2020-03-19T20:58:26.107 INFO:root:teuthology version: 1.0.0-293e8c9 2020-03-19T20:58:26.108 DEBUG:teuthology.run:Teuthology command: teuthology /tmp/teuthology-worker.EfDK5e.tmp -- --verbose --lock --description rados/singleton-nomsgr/{all/cache-fs-trunc.yaml mon_election/connectivity.yaml rados.yaml supported-random-distro$/{rhel_8.yaml}} --name sage-2020-03-19_17:25:48-rados-wip-sage-testing-2020-03-19-0855-distro-basic-smithi --block --owner scheduled_sage@teuthology --archive /home/teuthworker/archive/sage-2020-03-19_17:25:48-rados-wip-sage-testing-2020-03-19-0855-distro-basic-smithi/4868524 2020-03-19T20:58:26.146 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2020-03-19T20:58:26.202 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/sage-2020-03-19_17:25:48-rados-wip-sage-testing-2020-03-19-0855-distro-basic-smithi/4868524 branch: wip-sage-testing-2020-03-19-0855 description: rados/singleton-nomsgr/{all/cache-fs-trunc.yaml mon_election/connectivity.yaml rados.yaml supported-random-distro$/{rhel_8.yaml}} email: null first_in_suite: false job_id: '4868524' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi name: sage-2020-03-19_17:25:48-rados-wip-sage-testing-2020-03-19-0855-distro-basic-smithi nuke-on-error: true openstack: - volumes: count: 3 size: 10 os_type: rhel os_version: '8.1' overrides: admin_socket: branch: wip-sage-testing-2020-03-19-0855 ceph: conf: global: mon election default strategy: 3 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: debug filestore: 20 debug journal: 20 debug ms: 20 debug osd: 25 osd debug verify cached snaps: true osd debug verify missing on start: true osd op queue: debug_random osd op queue cut off: debug_random log-whitelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) sha1: 787db24c312ed9d310cd3f194b00548ede9a199c ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 install: ceph: sha1: 787db24c312ed9d310cd3f194b00548ede9a199c selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 workunit: branch: wip-sage-testing-2020-03-19-0855 sha1: 787db24c312ed9d310cd3f194b00548ede9a199c owner: scheduled_sage@teuthology priority: 90 repo: git://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - osd.0 - osd.1 - osd.2 - client.0 - client.1 sha1: 787db24c312ed9d310cd3f194b00548ede9a199c suite: rados suite_branch: wip-sage-testing-2020-03-19-0855 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_wip-sage-testing-2020-03-19-0855/qa suite_relpath: qa suite_repo: git://git.ceph.com/ceph-ci.git suite_sha1: 787db24c312ed9d310cd3f194b00548ede9a199c tasks: - install: null - ceph: conf: global: debug client: 20 debug mds: 20 debug ms: 1 osd max object name len: 460 osd max object namespace len: 64 log-whitelist: - overall HEALTH_ - \(CACHE_POOL_NO_HIT_SET\) pre-mgr-commands: - sudo ceph config set mgr mgr/devicehealth/enable_monitoring false --force - exec: client.0: - ceph osd pool create data_cache 4 - ceph osd tier add cephfs_data data_cache - ceph osd tier cache-mode data_cache writeback - ceph osd tier set-overlay cephfs_data data_cache - ceph osd pool set data_cache hit_set_type bloom - ceph osd pool set data_cache hit_set_count 8 - ceph osd pool set data_cache hit_set_period 3600 - ceph osd pool set data_cache min_read_recency_for_promote 0 - ceph-fuse: null - exec: client.0: - sudo chmod 777 $TESTDIR/mnt.0/ - dd if=/dev/urandom of=$TESTDIR/mnt.0/foo bs=1M count=5 - ls -al $TESTDIR/mnt.0/foo - truncate --size 0 $TESTDIR/mnt.0/foo - ls -al $TESTDIR/mnt.0/foo - dd if=/dev/urandom of=$TESTDIR/mnt.0/foo bs=1M count=5 - ls -al $TESTDIR/mnt.0/foo - cp $TESTDIR/mnt.0/foo /tmp/foo - sync - rados -p data_cache ls - - sleep 10 - rados -p data_cache ls - - rados -p data_cache cache-flush-evict-all - rados -p data_cache ls - - sleep 1 - exec: client.1: - hexdump -C /tmp/foo | head - hexdump -C $TESTDIR/mnt.1/foo | head - cmp $TESTDIR/mnt.1/foo /tmp/foo teuthology_branch: master tube: smithi verbose: false worker_log: /home/teuthworker/archive/worker_logs/worker.smithi.16493 2020-03-19T20:58:26.202 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_wip-sage-testing-2020-03-19-0855/qa; will attempt to use it 2020-03-19T20:58:26.202 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_wip-sage-testing-2020-03-19-0855/qa/tasks 2020-03-19T20:58:26.203 INFO:teuthology.run_tasks:Running task internal.check_packages... 2020-03-19T20:58:26.204 INFO:teuthology.task.internal:Checking packages... 2020-03-19T20:58:26.218 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'basic' and ceph hash '787db24c312ed9d310cd3f194b00548ede9a199c' 2020-03-19T20:58:26.219 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2020-03-19T20:58:26.219 INFO:teuthology.packaging:ref: None 2020-03-19T20:58:26.219 INFO:teuthology.packaging:tag: None 2020-03-19T20:58:26.219 INFO:teuthology.packaging:branch: wip-sage-testing-2020-03-19-0855 2020-03-19T20:58:26.219 INFO:teuthology.packaging:sha1: 787db24c312ed9d310cd3f194b00548ede9a199c 2020-03-19T20:58:26.220 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-sage-testing-2020-03-19-0855 2020-03-19T20:58:26.461 INFO:teuthology.task.internal:Found packages for ceph version 15.1.1-234.g787db24 2020-03-19T20:58:26.461 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2020-03-19T20:58:26.471 INFO:teuthology.task.internal:no buildpackages task found 2020-03-19T20:58:26.471 INFO:teuthology.run_tasks:Running task internal.lock_machines... 2020-03-19T20:58:26.485 INFO:teuthology.task.internal.lock_machines:Locking machines... 2020-03-19T20:58:26.485 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2020-03-19T20:58:26.534 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 1, have 2)... 2020-03-19T20:58:36.544 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 1, have 2)... 2020-03-19T20:58:46.561 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 1, have 2)... 2020-03-19T20:58:56.577 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 1, have 2)... 2020-03-19T20:59:06.590 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 1, have 2)... 2020-03-19T20:59:16.609 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 1, have 2)... 2020-03-19T20:59:26.626 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 1, have 2)... 2020-03-19T20:59:36.635 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 1, have 0)... 2020-03-19T20:59:46.654 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 1, have 0)... 2020-03-19T20:59:56.667 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 1, have 0)... 2020-03-19T21:00:06.683 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 1, have 0)... 2020-03-19T21:00:16.698 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 1, have 0)... 2020-03-19T21:00:26.718 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 1, have 0)... 2020-03-19T21:00:36.735 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 1, have 2)... 2020-03-19T21:00:46.753 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 1, have 2)... 2020-03-19T21:00:56.773 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 1, have 2)... 2020-03-19T21:01:06.796 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 1, have 2)... 2020-03-19T21:01:16.810 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 1, have 2)... 2020-03-19T21:01:26.828 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 1, have 2)... 2020-03-19T21:01:36.844 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 1, have 2)... 2020-03-19T21:01:46.861 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 1, have 2)... 2020-03-19T21:01:56.880 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 1, have 2)... 2020-03-19T21:02:06.897 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 1, have 2)... 2020-03-19T21:02:16.914 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 1, have 2)... 2020-03-19T21:02:26.930 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 1, have 2)... 2020-03-19T21:02:36.945 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 1, have 2)... 2020-03-19T21:02:46.962 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 1, have 2)... 2020-03-19T21:02:56.978 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 1, have 2)... 2020-03-19T21:03:06.993 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 1, have 2)... 2020-03-19T21:03:17.010 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 1, have 2)... 2020-03-19T21:03:27.026 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 1, have 2)... 2020-03-19T21:03:37.043 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 1, have 2)... 2020-03-19T21:03:47.059 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 1, have 2)... 2020-03-19T21:03:57.075 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 1, have 2)... 2020-03-19T21:04:07.084 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 1, have 1)... 2020-03-19T21:04:17.102 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 1, have 1)... 2020-03-19T21:04:27.120 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 1, have 1)... 2020-03-19T21:04:37.136 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 1, have 1)... 2020-03-19T21:04:47.153 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 1, have 1)... 2020-03-19T21:04:57.169 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 1, have 1)... 2020-03-19T21:05:07.185 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 1, have 1)... 2020-03-19T21:05:17.202 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 1, have 2)... 2020-03-19T21:05:27.214 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 1, have 2)... 2020-03-19T21:05:37.231 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 1, have 2)... 2020-03-19T21:05:47.246 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 1, have 2)... 2020-03-19T21:05:57.265 DEBUG:teuthology.lock.ops:lock_many request: {'count': 1, 'locked_by': 'scheduled_sage@teuthology', 'description': '/home/teuthworker/archive/sage-2020-03-19_17:25:48-rados-wip-sage-testing-2020-03-19-0855-distro-basic-smithi/4868524', 'machine_type': 'smithi'} 2020-03-19T21:05:57.291 DEBUG:teuthology.lock.ops:locked ubuntu@smithi170.front.sepia.ceph.com 2020-03-19T21:05:57.322 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=, signals=[15]) 2020-03-19T21:05:57.521 INFO:teuthology.provision.fog.smithi170:Scheduling deploy of rhel 8.1 2020-03-19T21:05:57.902 INFO:teuthology.orchestra.console:Power off smithi170 2020-03-19T21:05:57.902 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi170.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2020-03-19T21:05:57.923 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi170.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-03-19T21:06:01.946 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi170.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-03-19T21:06:06.058 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi170.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-03-19T21:06:06.266 INFO:teuthology.orchestra.console:Power off for smithi170 completed 2020-03-19T21:06:06.367 INFO:teuthology.orchestra.console:Power on smithi170 2020-03-19T21:06:06.367 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi170.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2020-03-19T21:06:06.386 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi170.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-03-19T21:06:10.407 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi170.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-03-19T21:06:14.518 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi170.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-03-19T21:06:14.770 INFO:teuthology.orchestra.console:Power on for smithi170 completed 2020-03-19T21:06:14.871 INFO:teuthology.provision.fog.smithi170:Waiting for deploy to finish 2020-03-19T21:08:45.891 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi170.front.sepia.ceph.com', 'timeout': 60} 2020-03-19T21:08:54.966 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi170.front.sepia.ceph.com', 'timeout': 60} 2020-03-19T21:09:04.026 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi170.front.sepia.ceph.com', 'timeout': 60} 2020-03-19T21:09:13.082 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi170.front.sepia.ceph.com', 'timeout': 60} 2020-03-19T21:09:22.140 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi170.front.sepia.ceph.com', 'timeout': 60} 2020-03-19T21:09:31.230 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi170.front.sepia.ceph.com', 'timeout': 60} 2020-03-19T21:09:40.318 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi170.front.sepia.ceph.com', 'timeout': 60} 2020-03-19T21:09:49.402 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi170.front.sepia.ceph.com', 'timeout': 60} 2020-03-19T21:10:55.404 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi170.front.sepia.ceph.com', 'timeout': 60} 2020-03-19T21:10:55.711 INFO:teuthology.orchestra.run.smithi170:> hostname 2020-03-19T21:10:56.026 INFO:teuthology.orchestra.run.smithi170.stdout:smithi170.front.sepia.ceph.com 2020-03-19T21:10:56.026 INFO:teuthology.orchestra.run.smithi170:> grep smithi170.front.sepia.ceph.com /etc/hosts 2020-03-19T21:10:56.081 INFO:teuthology.orchestra.run.smithi170.stdout:172.21.15.170 smithi170.front.sepia.ceph.com smithi170 2020-03-19T21:10:56.081 INFO:teuthology.orchestra.run.smithi170:> sudo hostname smithi170 2020-03-19T21:10:56.197 INFO:teuthology.orchestra.run.smithi170:> sudo sed -i -e 's/smithi170.front.sepia.ceph.com/smithi170/g' /etc/hosts 2020-03-19T21:10:56.225 INFO:teuthology.orchestra.run.smithi170:> sudo sed -i -e 's/172.21.15.170/172.21.15.170/g' /etc/hosts 2020-03-19T21:10:56.292 INFO:teuthology.orchestra.run.smithi170:> sudo sed -i -e 's/smithi170.front.sepia.ceph.com/smithi170/g' /etc/hostname 2020-03-19T21:10:56.359 INFO:teuthology.orchestra.run.smithi170:> sudo hostname smithi170 2020-03-19T21:10:56.425 INFO:teuthology.provision.fog.smithi170:Deploy complete! 2020-03-19T21:10:56.426 DEBUG:teuthology.parallel:result is None 2020-03-19T21:10:56.528 INFO:teuthology.lock.ops:Checking smithi170.front.sepia.ceph.com 2020-03-19T21:10:56.529 INFO:teuthology.lock.ops:New key found. Updating... 2020-03-19T21:10:56.548 INFO:teuthology.lock.ops:Updating [smithi170.front.sepia.ceph.com]: set os type and version on server 2020-03-19T21:10:56.549 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-03-19T21:10:56.550 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi170.front.sepia.ceph.com', 'timeout': 60} 2020-03-19T21:10:56.618 INFO:teuthology.orchestra.run.smithi170:> true 2020-03-19T21:10:56.783 INFO:teuthology.orchestra.run.smithi170:> uname -m 2020-03-19T21:10:56.837 INFO:teuthology.orchestra.run.smithi170.stdout:x86_64 2020-03-19T21:10:56.838 INFO:teuthology.orchestra.run.smithi170:> cat /etc/os-release 2020-03-19T21:10:56.891 INFO:teuthology.orchestra.run.smithi170.stdout:NAME="Red Hat Enterprise Linux" 2020-03-19T21:10:56.891 INFO:teuthology.orchestra.run.smithi170.stdout:VERSION="8.1 (Ootpa)" 2020-03-19T21:10:56.891 INFO:teuthology.orchestra.run.smithi170.stdout:ID="rhel" 2020-03-19T21:10:56.891 INFO:teuthology.orchestra.run.smithi170.stdout:ID_LIKE="fedora" 2020-03-19T21:10:56.891 INFO:teuthology.orchestra.run.smithi170.stdout:VERSION_ID="8.1" 2020-03-19T21:10:56.892 INFO:teuthology.orchestra.run.smithi170.stdout:PLATFORM_ID="platform:el8" 2020-03-19T21:10:56.892 INFO:teuthology.orchestra.run.smithi170.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.1 (Ootpa)" 2020-03-19T21:10:56.892 INFO:teuthology.orchestra.run.smithi170.stdout:ANSI_COLOR="0;31" 2020-03-19T21:10:56.892 INFO:teuthology.orchestra.run.smithi170.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.1:GA" 2020-03-19T21:10:56.892 INFO:teuthology.orchestra.run.smithi170.stdout:HOME_URL="https://www.redhat.com/" 2020-03-19T21:10:56.892 INFO:teuthology.orchestra.run.smithi170.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2020-03-19T21:10:56.892 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:10:56.892 INFO:teuthology.orchestra.run.smithi170.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2020-03-19T21:10:56.893 INFO:teuthology.orchestra.run.smithi170.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.1 2020-03-19T21:10:56.893 INFO:teuthology.orchestra.run.smithi170.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2020-03-19T21:10:56.893 INFO:teuthology.orchestra.run.smithi170.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.1" 2020-03-19T21:10:56.894 INFO:teuthology.lock.ops:Updating smithi170.front.sepia.ceph.com on lock server 2020-03-19T21:10:56.907 INFO:teuthology.task.internal.lock_machines:1 smithi machines locked this try, 1/1 locked so far 2020-03-19T21:10:56.924 INFO:teuthology.task.internal.lock_machines:Locked targets: smithi170.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCy6Zubi7wlI+ftilAaKFs+a8x8iX8tOxNsMUGMaJfZVLRO2JNZJfD6+TZ/frqrSjdEmDDBFcR+LPdkOZ22N0T/b1DxeUxSs5KNrf3KBKUWJIQ4O4Hvm1JTtOseX52/1//e9TDD6uaGSYV8Ge0Gj6SvpUe6Tz5oMI5MlOqXQ5vP86QWpFCifq2Ms24866n8IK4aruHvrwbtfu9fQzd1gnb8eYN/JqQriLJhKnwI16rz86b9ZHx/mWD8EPB40NuLLKO7W0uXg08I8SA/uQ0kfMHSC1eecF+OqOaP700QNaCQl4IEQJRmWzrGh5W6BGoyJDk/PKmzhc+KREQwrO1qIKZjUEp3Jcu81uR6PuBnr3l+pM34e5KWOjSBt2SOkIMnA4y/2hu1WxzJfB6L+5OSYL7lsZbANz8TZ3LfsKJmeFVnZIZKncXb9o8ZIQ9t8acfJBI8uqaBV4SCszLhPFRqrBY03EDb4f/sFXjt+Y8Rxnf08XbL7pyaKTraKgW9HZ9Osh8= 2020-03-19T21:10:56.924 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2020-03-19T21:10:56.967 INFO:teuthology.run_tasks:Running task internal.save_config... 2020-03-19T21:10:56.972 INFO:teuthology.task.internal:Saving configuration 2020-03-19T21:10:56.987 INFO:teuthology.run_tasks:Running task internal.check_lock... 2020-03-19T21:10:56.997 INFO:teuthology.task.internal.check_lock:Checking locks... 2020-03-19T21:10:57.013 DEBUG:teuthology.task.internal.check_lock:machine status is {u'is_vm': False, u'locked': True, u'locked_since': u'2020-03-19 21:05:57.281653', u'locked_by': u'scheduled_sage@teuthology', u'up': True, u'mac_address': None, u'name': u'smithi170.front.sepia.ceph.com', u'os_version': u'8.1', u'machine_type': u'smithi', u'vm_host': None, u'os_type': u'rhel', u'arch': u'x86_64', u'ssh_pub_key': u'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOwwmdgAuZebALWSYR2L81gAlvZf5NU23/Bt65k57+WTl92Iy15Lzmpc9CiFbuNm7q5iFBec/+rUwm7e+UTXIsE=', u'description': u'/home/teuthworker/archive/sage-2020-03-19_17:25:48-rados-wip-sage-testing-2020-03-19-0855-distro-basic-smithi/4868524'} 2020-03-19T21:10:57.013 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2020-03-19T21:10:57.024 INFO:teuthology.task.internal:roles: ubuntu@smithi170.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'osd.0', 'osd.1', 'osd.2', 'client.0', 'client.1'] 2020-03-19T21:10:57.025 INFO:teuthology.run_tasks:Running task console_log... 2020-03-19T21:10:57.066 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=, signals=[15]) 2020-03-19T21:10:57.067 INFO:teuthology.run_tasks:Running task internal.connect... 2020-03-19T21:10:57.079 INFO:teuthology.task.internal:Opening connections... 2020-03-19T21:10:57.079 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi170.front.sepia.ceph.com 2020-03-19T21:10:57.080 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi170.front.sepia.ceph.com', 'timeout': 60} 2020-03-19T21:10:57.148 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2020-03-19T21:10:57.160 INFO:teuthology.orchestra.run.smithi170:> uname -m 2020-03-19T21:10:57.308 INFO:teuthology.orchestra.run.smithi170.stdout:x86_64 2020-03-19T21:10:57.309 INFO:teuthology.orchestra.run.smithi170:> cat /etc/os-release 2020-03-19T21:10:57.362 INFO:teuthology.orchestra.run.smithi170.stdout:NAME="Red Hat Enterprise Linux" 2020-03-19T21:10:57.362 INFO:teuthology.orchestra.run.smithi170.stdout:VERSION="8.1 (Ootpa)" 2020-03-19T21:10:57.362 INFO:teuthology.orchestra.run.smithi170.stdout:ID="rhel" 2020-03-19T21:10:57.362 INFO:teuthology.orchestra.run.smithi170.stdout:ID_LIKE="fedora" 2020-03-19T21:10:57.362 INFO:teuthology.orchestra.run.smithi170.stdout:VERSION_ID="8.1" 2020-03-19T21:10:57.362 INFO:teuthology.orchestra.run.smithi170.stdout:PLATFORM_ID="platform:el8" 2020-03-19T21:10:57.363 INFO:teuthology.orchestra.run.smithi170.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.1 (Ootpa)" 2020-03-19T21:10:57.363 INFO:teuthology.orchestra.run.smithi170.stdout:ANSI_COLOR="0;31" 2020-03-19T21:10:57.363 INFO:teuthology.orchestra.run.smithi170.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.1:GA" 2020-03-19T21:10:57.363 INFO:teuthology.orchestra.run.smithi170.stdout:HOME_URL="https://www.redhat.com/" 2020-03-19T21:10:57.364 INFO:teuthology.orchestra.run.smithi170.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2020-03-19T21:10:57.364 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:10:57.364 INFO:teuthology.orchestra.run.smithi170.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2020-03-19T21:10:57.364 INFO:teuthology.orchestra.run.smithi170.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.1 2020-03-19T21:10:57.365 INFO:teuthology.orchestra.run.smithi170.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2020-03-19T21:10:57.365 INFO:teuthology.orchestra.run.smithi170.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.1" 2020-03-19T21:10:57.365 INFO:teuthology.lock.ops:Updating smithi170.front.sepia.ceph.com on lock server 2020-03-19T21:10:57.381 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2020-03-19T21:10:57.424 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2020-03-19T21:10:57.434 INFO:teuthology.task.internal:Checking for old test directory... 2020-03-19T21:10:57.435 INFO:teuthology.orchestra.run.smithi170:> test '!' -e /home/ubuntu/cephtest 2020-03-19T21:10:57.448 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2020-03-19T21:10:57.459 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2020-03-19T21:10:57.459 INFO:teuthology.orchestra.run.smithi170:> test -z $(ls -A /var/lib/ceph) 2020-03-19T21:10:57.514 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2020-03-19T21:10:57.544 INFO:teuthology.run_tasks:Running task kernel... 2020-03-19T21:10:57.566 DEBUG:teuthology.task.kernel:normalized config {'mon.a': {'sha1': 'distro', 'kdb': True}, 'client.0': {'sha1': 'distro', 'kdb': True}, 'mds.a': {'sha1': 'distro', 'kdb': True}, 'osd.1': {'sha1': 'distro', 'kdb': True}, 'osd.0': {'sha1': 'distro', 'kdb': True}, 'client.1': {'sha1': 'distro', 'kdb': True}, 'osd.2': {'sha1': 'distro', 'kdb': True}, 'mgr.x': {'sha1': 'distro', 'kdb': True}} 2020-03-19T21:10:57.567 INFO:teuthology.task.kernel:config {'mon.a': {'sha1': 'distro', 'kdb': True}}, timeout 300 2020-03-19T21:10:57.567 INFO:teuthology.orchestra.run.smithi170:> uname -r 2020-03-19T21:10:57.581 INFO:teuthology.orchestra.run.smithi170.stdout:4.18.0-147.5.1.el8_1.x86_64 2020-03-19T21:10:57.581 INFO:teuthology.task.kernel:Running kernel on smithi170: 4.18.0-147.5.1.el8_1.x86_64 2020-03-19T21:10:57.582 INFO:teuthology.orchestra.run.smithi170:> sudo yum install -y kernel 2020-03-19T21:10:59.270 INFO:teuthology.orchestra.run.smithi170.stdout:Updating Subscription Management repositories. 2020-03-19T21:10:59.270 INFO:teuthology.orchestra.run.smithi170.stdout:Unable to read consumer identity 2020-03-19T21:10:59.383 INFO:teuthology.orchestra.run.smithi170.stdout:This system is not registered to Red Hat Subscription Management. You can use subscription-manager to register. 2020-03-19T21:11:00.500 INFO:teuthology.orchestra.run.smithi170.stdout:Copr repo for ceph-el8 owned by ktdreyer 284 kB/s | 38 kB 00:00 2020-03-19T21:11:00.748 INFO:teuthology.orchestra.run.smithi170.stdout:Extra Packages for Enterprise Linux 28 MB/s | 6.1 MB 00:00 2020-03-19T21:11:01.919 INFO:teuthology.orchestra.run.smithi170.stdout:lab-extras 5.6 MB/s | 30 kB 00:00 2020-03-19T21:11:02.180 INFO:teuthology.orchestra.run.smithi170.stderr:Modular dependency problem: 2020-03-19T21:11:02.180 INFO:teuthology.orchestra.run.smithi170.stderr: 2020-03-19T21:11:02.180 INFO:teuthology.orchestra.run.smithi170.stderr: Problem: conflicting requests 2020-03-19T21:11:02.181 INFO:teuthology.orchestra.run.smithi170.stderr: - nothing provides module(perl:5.26) needed by module perl-YAML:1.24:8010020190322131811:a5949e2e-0.x86_64 2020-03-19T21:11:02.380 INFO:teuthology.orchestra.run.smithi170.stdout:Package kernel-4.18.0-147.el8.x86_64 is already installed. 2020-03-19T21:11:02.380 INFO:teuthology.orchestra.run.smithi170.stdout:Package kernel-4.18.0-147.5.1.el8_1.x86_64 is already installed. 2020-03-19T21:11:02.461 INFO:teuthology.orchestra.run.smithi170.stdout:Dependencies resolved. 2020-03-19T21:11:02.462 INFO:teuthology.orchestra.run.smithi170.stdout:Nothing to do. 2020-03-19T21:11:02.463 INFO:teuthology.orchestra.run.smithi170.stdout:Complete! 2020-03-19T21:11:02.577 INFO:teuthology.orchestra.run.smithi170:> echo no | sudo yum reinstall kernel || true 2020-03-19T21:11:02.902 INFO:teuthology.orchestra.run.smithi170.stdout:Updating Subscription Management repositories. 2020-03-19T21:11:02.902 INFO:teuthology.orchestra.run.smithi170.stdout:Unable to read consumer identity 2020-03-19T21:11:03.031 INFO:teuthology.orchestra.run.smithi170.stdout:This system is not registered to Red Hat Subscription Management. You can use subscription-manager to register. 2020-03-19T21:11:03.180 INFO:teuthology.orchestra.run.smithi170.stdout:Last metadata expiration check: 0:00:02 ago on Thu 19 Mar 2020 09:11:01 PM UTC. 2020-03-19T21:11:03.207 INFO:teuthology.orchestra.run.smithi170.stderr:Modular dependency problem: 2020-03-19T21:11:03.208 INFO:teuthology.orchestra.run.smithi170.stderr: 2020-03-19T21:11:03.208 INFO:teuthology.orchestra.run.smithi170.stderr: Problem: conflicting requests 2020-03-19T21:11:03.208 INFO:teuthology.orchestra.run.smithi170.stderr: - nothing provides module(perl:5.26) needed by module perl-YAML:1.24:8010020190322131811:a5949e2e-0.x86_64 2020-03-19T21:11:03.287 INFO:teuthology.orchestra.run.smithi170.stdout:Installed package kernel-4.18.0-147.el8.x86_64 (from anaconda) not available. 2020-03-19T21:11:03.287 INFO:teuthology.orchestra.run.smithi170.stdout:Installed package kernel-4.18.0-147.5.1.el8_1.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2020-03-19T21:11:03.338 INFO:teuthology.orchestra.run.smithi170.stderr:Error: No packages marked for reinstall. 2020-03-19T21:11:03.403 INFO:teuthology.orchestra.run.smithi170:> sudo yum reinstall -y kernel || true 2020-03-19T21:11:03.728 INFO:teuthology.orchestra.run.smithi170.stdout:Updating Subscription Management repositories. 2020-03-19T21:11:03.728 INFO:teuthology.orchestra.run.smithi170.stdout:Unable to read consumer identity 2020-03-19T21:11:03.833 INFO:teuthology.orchestra.run.smithi170.stdout:This system is not registered to Red Hat Subscription Management. You can use subscription-manager to register. 2020-03-19T21:11:03.977 INFO:teuthology.orchestra.run.smithi170.stdout:Last metadata expiration check: 0:00:02 ago on Thu 19 Mar 2020 09:11:01 PM UTC. 2020-03-19T21:11:04.004 INFO:teuthology.orchestra.run.smithi170.stderr:Modular dependency problem: 2020-03-19T21:11:04.005 INFO:teuthology.orchestra.run.smithi170.stderr: 2020-03-19T21:11:04.005 INFO:teuthology.orchestra.run.smithi170.stderr: Problem: conflicting requests 2020-03-19T21:11:04.005 INFO:teuthology.orchestra.run.smithi170.stderr: - nothing provides module(perl:5.26) needed by module perl-YAML:1.24:8010020190322131811:a5949e2e-0.x86_64 2020-03-19T21:11:04.112 INFO:teuthology.orchestra.run.smithi170.stdout:Installed package kernel-4.18.0-147.el8.x86_64 (from anaconda) not available. 2020-03-19T21:11:04.112 INFO:teuthology.orchestra.run.smithi170.stdout:Installed package kernel-4.18.0-147.5.1.el8_1.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2020-03-19T21:11:04.153 INFO:teuthology.orchestra.run.smithi170.stderr:Error: No packages marked for reinstall. 2020-03-19T21:11:04.208 INFO:teuthology.orchestra.run.smithi170:> rpm -q kernel --last | head -n 1 2020-03-19T21:11:04.258 INFO:teuthology.orchestra.run.smithi170.stdout:kernel-4.18.0-147.5.1.el8_1.x86_64 Mon 16 Mar 2020 03:25:53 PM UTC 2020-03-19T21:11:04.260 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-147.5.1.el8_1.x86_64 2020-03-19T21:11:04.260 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2020-03-19T21:11:04.260 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2020-03-19T21:11:04.260 INFO:teuthology.orchestra.run.smithi170:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2020-03-19T21:11:04.291 INFO:teuthology.orchestra.run.smithi170.stdout:ttyS1 2020-03-19T21:11:04.308 INFO:teuthology.run_tasks:Running task internal.base... 2020-03-19T21:11:04.318 INFO:teuthology.task.internal:Creating test directory... 2020-03-19T21:11:04.319 INFO:teuthology.orchestra.run.smithi170:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2020-03-19T21:11:04.361 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2020-03-19T21:11:04.372 INFO:teuthology.run_tasks:Running task internal.archive... 2020-03-19T21:11:04.384 INFO:teuthology.task.internal:Creating archive directory... 2020-03-19T21:11:04.385 INFO:teuthology.orchestra.run.smithi170:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2020-03-19T21:11:04.425 INFO:teuthology.run_tasks:Running task internal.coredump... 2020-03-19T21:11:04.462 INFO:teuthology.task.internal:Enabling coredump saving... 2020-03-19T21:11:04.462 INFO:teuthology.orchestra.run.smithi170:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2020-03-19T21:11:04.524 INFO:teuthology.orchestra.run.smithi170.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-03-19T21:11:04.538 INFO:teuthology.orchestra.run.smithi170.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-03-19T21:11:04.539 INFO:teuthology.run_tasks:Running task internal.sudo... 2020-03-19T21:11:04.554 INFO:teuthology.task.internal:Configuring sudo... 2020-03-19T21:11:04.554 INFO:teuthology.orchestra.run.smithi170:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2020-03-19T21:11:04.582 INFO:teuthology.run_tasks:Running task internal.syslog... 2020-03-19T21:11:04.593 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2020-03-19T21:11:04.594 INFO:teuthology.orchestra.run.smithi170:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2020-03-19T21:11:04.636 INFO:teuthology.orchestra.run.smithi170:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2020-03-19T21:11:04.707 INFO:teuthology.orchestra.run.smithi170:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2020-03-19T21:11:04.780 INFO:teuthology.orchestra.run.smithi170:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2020-03-19T21:11:04.850 INFO:teuthology.orchestra.run.smithi170:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2020-03-19T21:11:04.902 INFO:teuthology.orchestra.run.smithi170:> sudo sh -c 'cat > /etc/rsyslog.d/80-cephtest.conf' 2020-03-19T21:11:04.988 INFO:teuthology.orchestra.run.smithi170:> sudo service rsyslog restart 2020-03-19T21:11:05.088 INFO:teuthology.orchestra.run.smithi170.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2020-03-19T21:11:05.532 INFO:teuthology.run_tasks:Running task internal.timer... 2020-03-19T21:11:05.542 INFO:teuthology.task.internal:Starting timer... 2020-03-19T21:11:05.542 INFO:teuthology.run_tasks:Running task pcp... 2020-03-19T21:11:05.628 INFO:teuthology.run_tasks:Running task selinux... 2020-03-19T21:11:05.644 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2020-03-19T21:11:05.644 INFO:teuthology.orchestra.run.smithi170:> sudo service auditd rotate 2020-03-19T21:11:05.720 INFO:teuthology.orchestra.run.smithi170.stdout:Rotating logs: [ OK ] 2020-03-19T21:11:05.722 DEBUG:teuthology.task.selinux:Getting current SELinux state 2020-03-19T21:11:05.722 INFO:teuthology.orchestra.run.smithi170:> /usr/sbin/getenforce 2020-03-19T21:11:05.739 INFO:teuthology.orchestra.run.smithi170.stdout:Permissive 2020-03-19T21:11:05.739 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi170.front.sepia.ceph.com': 'permissive'} 2020-03-19T21:11:05.740 INFO:teuthology.orchestra.run.smithi170:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av '\(comm="dmidecode"\|chronyd.service\|name="cephtest"\|scontext=system_u:system_r:nrpe_t:s0\|scontext=system_u:system_r:pcp_pmlogger_t\|scontext=system_u:system_r:pcp_pmcd_t:s0\|comm="rhsmd"\|scontext=system_u:system_r:syslogd_t:s0\|tcontext=system_u:system_r:nrpe_t:s0\|comm="updatedb"\|comm="smartd"\|comm="rhsmcertd-worke"\|comm="setroubleshootd"\|comm="rpm"\|tcontext=system_u:object_r:container_runtime_exec_t:s0\|scontext=system_u:system_r:logrotate_t:s0\)' 2020-03-19T21:11:05.808 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-03-19T21:11:05.808 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2020-03-19T21:11:05.809 INFO:teuthology.orchestra.run.smithi170:> sudo /usr/sbin/setenforce permissive 2020-03-19T21:11:05.894 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2020-03-19T21:11:05.910 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master was just updated; assuming it is current 2020-03-19T21:11:05.910 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master to branch origin/master 2020-03-19T21:11:05.927 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}] 2020-03-19T21:11:05.931 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi170.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/cephlab.yml 2020-03-19T21:11:06.300 INFO:teuthology.task.ansible.out:Using /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/ansible.cfg as config file 2020-03-19T21:11:07.679 INFO:teuthology.task.ansible.out: [WARNING]: file /home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/roles/testnode/tasks/setup-opensuse.yml is empty and had no tasks to include 2020-03-19T21:11:11.454 INFO:teuthology.task.ansible.out: PLAY [all] ********************************************************************* 2020-03-19T21:11:11.501 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:11:11 +0000 (0:00:00.104) 0:00:00.104 ******** 2020-03-19T21:11:14.204 INFO:teuthology.task.ansible.out: TASK [Gathering Facts] ********************************************************* 2020-03-19T21:11:14.205 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] 2020-03-19T21:11:14.340 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:11:14 +0000 (0:00:02.838) 0:00:02.943 ******** 2020-03-19T21:11:14.792 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the sudo group.] ******************************** 2020-03-19T21:11:14.793 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => {"changed": false, "gid": 1000, "name": "sudo", "state": "present", "system": false} 2020-03-19T21:11:14.855 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:11:14 +0000 (0:00:00.515) 0:00:03.459 ******** 2020-03-19T21:11:15.549 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the ansible user.] ****************************** 2020-03-19T21:11:15.550 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 1100, "groups": "sudo", "home": "/home/cm", "move_home": false, "name": "cm", "shell": "/bin/bash", "state": "present", "uid": 1100} 2020-03-19T21:11:15.591 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:11:15 +0000 (0:00:00.735) 0:00:04.194 ******** 2020-03-19T21:11:15.627 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Delete the ansible users password.] ******************** 2020-03-19T21:11:15.628 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:15.681 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:11:15 +0000 (0:00:00.089) 0:00:04.284 ******** 2020-03-19T21:11:16.060 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Ensure includedir is present in sudoers.] ************** ok: [smithi170.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-03-19T21:11:16.118 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:11:16 +0000 (0:00:00.437) 0:00:04.722 ******** 2020-03-19T21:11:17.202 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the cephlab_sudo sudoers.d file.] *************** 2020-03-19T21:11:17.203 INFO:teuthology.task.ansible.out:changed: [smithi170.front.sepia.ceph.com] => {"changed": true, "checksum": "6c8e8b7ecc895eb2094cafe75c34b58f183442a7", "dest": "/etc/sudoers.d/cephlab_sudo", "gid": 0, "group": "root", "md5sum": "c3657b9ef874d0ad278742c3210136a0", "mode": "0440", "owner": "root", "secontext": "system_u:object_r:etc_t:s0", "size": 182, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1584652276.15-98018380982903/source", "state": "file", "uid": 0} 2020-03-19T21:11:17.245 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:11:17 +0000 (0:00:01.126) 0:00:05.848 ******** 2020-03-19T21:11:17.954 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Add authorized keys for the ansible user.] ************* 2020-03-19T21:11:17.955 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => {"changed": false, "comment": null, "exclusive": true, "follow": false, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv 2020-03-19T21:11:17.957 INFO:teuthology.task.ansible.out:4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2020-03-19T21:11:18.083 INFO:teuthology.task.ansible.out: PLAY [teuthology] ************************************************************** 2020-03-19T21:11:18.084 INFO:teuthology.task.ansible.out:skipping: no hosts matched 2020-03-19T21:11:18.088 INFO:teuthology.task.ansible.out: PLAY [testnodes] *************************************************************** 2020-03-19T21:11:18.303 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:11:18 +0000 (0:00:01.057) 0:00:06.906 ******** 2020-03-19T21:11:18.972 INFO:teuthology.task.ansible.out: TASK [Gathering Facts] ********************************************************* 2020-03-19T21:11:18.974 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] 2020-03-19T21:11:19.111 INFO:teuthology.task.ansible.out: [WARNING]: Using run_once with the free strategy is not currently supported. This task will still be executed for every host in the inventory list. 2020-03-19T21:11:19.112 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:11:19 +0000 (0:00:00.808) 0:00:07.715 ******** 2020-03-19T21:11:19.690 INFO:teuthology.task.ansible.out: TASK [users : Check keys_repo HEAD sha1] *************************************** 2020-03-19T21:11:19.690 INFO:teuthology.task.ansible.out:changed: [smithi170.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "git ls-remote https://github.com/ceph/keys HEAD | awk '{ print $1 }'", "delta": "0:00:00.257718", "end": "2020-03-19 21:11:19.653973", "rc": 0, "start": "2020-03-19 21:11:19.396255", "stderr": "", "stderr_lines": [], "stdout": "b22ad6dbd27eb2f944823eb8be286527dd352386", "stdout_lines": ["b22ad6dbd27eb2f944823eb8be286527dd352386"]} 2020-03-19T21:11:19.751 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:11:19 +0000 (0:00:00.639) 0:00:08.354 ******** 2020-03-19T21:11:19.965 INFO:teuthology.task.ansible.out: TASK [users : Check host's /keys-repo-sha1 sentinel file] ********************** 2020-03-19T21:11:19.966 INFO:teuthology.task.ansible.out:changed: [smithi170.front.sepia.ceph.com] => {"changed": true, "cmd": ["cat", "/keys-repo-sha1"], "delta": "0:00:00.009364", "end": "2020-03-19 21:11:19.947313", "failed_when_result": false, "rc": 0, "start": "2020-03-19 21:11:19.937949", "stderr": "", "stderr_lines": [], "stdout": "b22ad6dbd27eb2f944823eb8be286527dd352386", "stdout_lines": ["b22ad6dbd27eb2f944823eb8be286527dd352386"]} 2020-03-19T21:11:20.025 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:11:20 +0000 (0:00:00.274) 0:00:08.628 ******** 2020-03-19T21:11:20.061 INFO:teuthology.task.ansible.out: TASK [users : Determine if we can skip users and pubkeys updates] ************** 2020-03-19T21:11:20.062 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => {"ansible_facts": {"perform_users_role": false}, "changed": false} 2020-03-19T21:11:20.116 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:11:20 +0000 (0:00:00.090) 0:00:08.719 ******** 2020-03-19T21:11:20.149 INFO:teuthology.task.ansible.out: TASK [users : Merge extra_admin_users into managed_admin_users] **************** skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:20.189 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:11:20 +0000 (0:00:00.073) 0:00:08.793 ******** 2020-03-19T21:11:20.215 INFO:teuthology.task.ansible.out: TASK [users : Remove managed_admin_users from managed_users] ******************* 2020-03-19T21:11:20.216 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:20.271 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:11:20 +0000 (0:00:00.081) 0:00:08.874 ******** 2020-03-19T21:11:20.308 INFO:teuthology.task.ansible.out: TASK [users : Filter the managed_users list] *********************************** 2020-03-19T21:11:20.309 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:20.349 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:11:20 +0000 (0:00:00.078) 0:00:08.952 ******** 2020-03-19T21:11:20.387 INFO:teuthology.task.ansible.out: TASK [users : Filter the managed_admin_users list] ***************************** 2020-03-19T21:11:20.388 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:20.433 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:11:20 +0000 (0:00:00.083) 0:00:09.036 ******** 2020-03-19T21:11:20.462 INFO:teuthology.task.ansible.out: TASK [users : Set UID_MIN to 1001] ********************************************* skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:20.512 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:11:20 +0000 (0:00:00.079) 0:00:09.116 ******** 2020-03-19T21:11:20.580 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ 2020-03-19T21:11:20.582 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={u'ovpn': u'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9', u'name': u'andrewschoen'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:20.592 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={u'ovpn': u'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n', u'name': u'zack'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:20.603 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={u'ovpn': u'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd', u'name': u'kdreyer'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:20.614 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={u'ovpn': u'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n', u'name': u'dmick'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:20.615 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={u'ovpn': u'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n', u'name': u'sage'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:20.626 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={u'ovpn': u'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n', u'name': u'dgalloway'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:20.637 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={u'ovpn': u'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045', u'name': u'akraitma'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:20.649 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={u'ovpn': u'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e', u'name': u'trociny'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:20.660 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={u'ovpn': u'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007', u'name': u'smithfarm', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:20.672 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={u'ovpn': u'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n', u'name': u'mbenjamin'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:20.674 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={u'ovpn': u'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01', u'name': u'aemerson', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt", "name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01"}, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:20.685 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={u'ovpn': u'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78', u'name': u'amaredia', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:20.696 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={u'ovpn': u'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5', u'name': u'tserlin', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:20.697 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={u'ovpn': u'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28', u'name': u'dis'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:20.708 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={u'ovpn': u'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n', u'name': u'gregf'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:20.734 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={u'ovpn': u'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n', u'name': u'joshd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi170.front.sepia.ceph.com] => (item={u'ovpn': u'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n', u'name': u'davidz'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi170.front.sepia.ceph.com] => (item={u'ovpn': u'gmeno@gmeno-virtual-machine FKFu8B2pMqotpmEVAO1few 8229574e499eaf767a408909f5afdf2e2a0bb8f3e61b18d63a651f7102c68dbc', u'name': u'gmeno'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gmeno", "ovpn": "gmeno@gmeno-virtual-machine FKFu8B2pMqotpmEVAO1few 8229574e499eaf767a408909f5afdf2e2a0bb8f3e61b18d63a651f7102c68dbc"}, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:20.744 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={u'ovpn': u'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1', u'name': u'ivancich'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:20.745 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={u'ovpn': u'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n', u'name': u'wusui'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:20.757 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={u'ovpn': u'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304', u'name': u'zyan'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:20.758 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={u'ovpn': u'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH 5ivdxgFO4eIkbXVhl8xkvw 59212d29b8b42d9fe457c1b2c43d774e1d25807be10dcc1252d4aec63b97a467\n', u'name': u'yuriw'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH 5ivdxgFO4eIkbXVhl8xkvw 59212d29b8b42d9fe457c1b2c43d774e1d25807be10dcc1252d4aec63b97a467\n"}, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:20.770 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={u'ovpn': u'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n', u'name': u'tamil'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:20.771 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={u'ovpn': u'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n', u'name': u'jowilkin'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:20.783 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={u'ovpn': u'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4', u'name': u'bhubbard'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:20.784 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={u'ovpn': u'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n', u'name': u'yehudasa'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:20.795 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={u'ovpn': u'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4', u'name': u'dang'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:20.806 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={u'ovpn': u'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb', u'name': u'branto'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:20.817 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={u'ovpn': u'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6', u'name': u'xiaoxichen'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:20.818 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={u'ovpn': u'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e', u'name': u'ffilz'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:20.829 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={u'ovpn': u'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n', u'name': u'joao'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:20.830 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={u'ovpn': u'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n', u'name': u'nhm'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:20.842 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={u'ovpn': u'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313', u'name': u'nwatkins'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:20.843 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={u'ovpn': u'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22', u'name': u'mkidd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:20.857 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={u'ovpn': u'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af', u'name': u'jlopez'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:20.857 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={u'ovpn': u'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb', u'name': u'haomaiwang'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:20.869 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={u'ovpn': u'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608', u'name': u'jdillaman'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:20.870 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={u'ovpn': u'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3', u'name': u'kchai'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:20.883 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={u'ovpn': u'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2', u'name': u'vumrao'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:20.885 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={u'ovpn': u'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n', u'name': u'dfuller'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:20.895 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={u'ovpn': u'owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n', u'name': u'owasserm'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "owasserm", "ovpn": "owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n"}, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:20.896 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={u'ovpn': u'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n', u'name': u'vasu'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:20.907 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={u'ovpn': u'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37', u'name': u'smohan'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:20.919 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={u'ovpn': u'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114', u'name': u'abhi'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:20.920 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={u'ovpn': u'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611', u'name': u'cbodley'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:20.932 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={u'ovpn': u'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195', u'name': u'mwatts'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "skip_reason": "Conditional result was False"} skipping: [smithi170.front.sepia.ceph.com] => (item={u'ovpn': u'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6', u'name': u'prsrivas'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:20.943 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={u'ovpn': u'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea', u'name': u'pdonnell'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:20.944 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={u'ovpn': u'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868', u'name': u'jlayton'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:20.956 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={u'ovpn': u'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f', u'name': u'rzarzynski'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:20.967 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={u'ovpn': u'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e', u'name': u'rdias'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:20.968 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={u'ovpn': u'vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953', u'name': u'vshankar'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953"}, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:20.980 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={u'ovpn': u'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58', u'name': u'akupczyk'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:20.992 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={u'ovpn': u'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e', u'name': u'nojha'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e"}, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:21.003 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={u'ovpn': u'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43', u'name': u'ifed01'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:21.004 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={u'ovpn': u'myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4', u'name': u'myoungwon'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4"}, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:21.015 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={u'ovpn': u'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2', u'name': u'gabrioux'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:21.015 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={u'ovpn': u'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f', u'name': u'leseb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:21.027 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={u'ovpn': u'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca', u'name': u'hchen'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:21.028 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={u'ovpn': u'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n', u'name': u'jcollin'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:21.038 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={u'ovpn': u'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc', u'name': u'xxg'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:21.050 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={u'ovpn': u'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53', u'name': u'pcuzner'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:21.050 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={u'ovpn': u'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2', u'name': u'mkogan'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:21.062 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={u'ovpn': u'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed', u'name': u'vumrao'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:21.063 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={u'ovpn': u'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736', u'name': u'henrix'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:21.074 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={u'ovpn': u'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26', u'name': u'pbs1108'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:21.076 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={u'ovpn': u'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315', u'name': u'clacroix'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:21.087 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={u'ovpn': u'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006', u'name': u'epuertat'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:21.088 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={u'ovpn': u'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461', u'name': u'tdehler'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:21.099 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={u'ovpn': u'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305', u'name': u'laura'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:21.099 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={u'ovpn': u'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce', u'name': u'yaarit'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:21.110 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={u'ovpn': u'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3', u'name': u'rpavani1998'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:21.111 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={u'ovpn': u'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4', u'name': u'rishabh'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:21.122 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={u'ovpn': u'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2', u'name': u'skrah'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:21.123 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={u'ovpn': u'smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8', u'name': u'smanjara'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8"}, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:21.135 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={u'ovpn': u'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41', u'name': u'bengland'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:21.137 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={u'ovpn': u'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d', u'name': u'pnawracay'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:21.148 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={u'ovpn': u'alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8', u'name': u'alfonsomthd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:21.149 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={u'ovpn': u'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e', u'name': u'oliveiradan'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:21.160 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={u'ovpn': u'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba', u'name': u'swagner'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:21.161 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={u'ovpn': u'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e', u'name': u'yuvalif'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e"}, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:21.172 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={u'ovpn': u'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852', u'name': u'kkeithle'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:21.172 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={u'ovpn': u'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130', u'name': u'emmericp'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:21.186 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={u'ovpn': u'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0', u'name': u'mchangir'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:21.186 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={u'ovpn': u'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286', u'name': u'sidharthanup'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:21.199 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={u'ovpn': u'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31', u'name': u'varsha'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:21.200 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={u'ovpn': u'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n', u'name': u'sjust'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:21.220 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={u'ovpn': u'deepika@x1carbon BgvhtHMrCcelT98ekPDMoA 6fa3cf08186b93b966b68841f3507a0809d348c5097b39b6140832cac2afd511', u'name': u'ideepika'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ideepika", "ovpn": "deepika@x1carbon BgvhtHMrCcelT98ekPDMoA 6fa3cf08186b93b966b68841f3507a0809d348c5097b39b6140832cac2afd511"}, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:21.222 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={u'ovpn': u'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46', u'name': u'gsalomon'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "skip_reason": "Conditional result was False"} skipping: [smithi170.front.sepia.ceph.com] => (item={u'ovpn': u'skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de', u'name': u'soumyakoduri'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:21.233 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={u'ovpn': u'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a', u'name': u'kyr'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:21.244 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={u'ovpn': u'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849', u'name': u'sseshasa'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:21.246 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={u'ovpn': u'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189', u'name': u'rfriedma'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:21.247 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={u'ovpn': u'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b', u'name': u'lmb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:21.258 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={u'ovpn': u'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc', u'name': u'ksirivad'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc"}, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:21.259 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={u'ovpn': u'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4', u'name': u'ofriedma'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:21.271 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={u'ovpn': u'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52', u'name': u'h4iresis'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:21.272 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={u'ovpn': u'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5', u'name': u'shyukri'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:21.290 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={u'ovpn': u'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e', u'name': u'scarvalhojr'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "skip_reason": "Conditional result was False"} skipping: [smithi170.front.sepia.ceph.com] => (item={u'ovpn': u'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b', u'name': u'rakesh'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:21.293 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={u'ovpn': u'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f', u'name': u'ryg-'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:21.304 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={u'ovpn': u'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3', u'name': u'prsrivas'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:21.305 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={u'ovpn': u'zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42', u'name': u'zdover'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zdover", "ovpn": "zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42"}, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:21.316 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={u'ovpn': u'rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b', u'name': u'rraja'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rraja", "ovpn": "rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b"}, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:21.317 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={u'ovpn': u'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f', u'name': u'toabctl'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:21.328 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={u'ovpn': u'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39', u'name': u'mgfritch'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:21.339 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={u'ovpn': u'xiubli@fedora1 vosB15bA3uALkBrmjcKFUg 4ef5cb132c30563fd5fb9e4e9352786ad0d2258e41ed63f91c1f65f69b54d50d', u'name': u'xiubli'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiubli", "ovpn": "xiubli@fedora1 vosB15bA3uALkBrmjcKFUg 4ef5cb132c30563fd5fb9e4e9352786ad0d2258e41ed63f91c1f65f69b54d50d"}, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:21.339 INFO:teuthology.task.ansible.out: 2020-03-19T21:11:21.341 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={u'ovpn': u'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f', u'name': u'jafaj'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:21.351 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={u'ovpn': u'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef', u'name': u'jschmid'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:21.362 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={u'ovpn': u'kachau@thinkpad GsZIGCdLojpvOZDxwcWSlg 0638220db65953e0cd1f134c499e4a713b32a882c8a92458afcb495dbd087518', u'name': u'kachau'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kachau", "ovpn": "kachau@thinkpad GsZIGCdLojpvOZDxwcWSlg 0638220db65953e0cd1f134c499e4a713b32a882c8a92458afcb495dbd087518"}, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:21.363 INFO:teuthology.task.ansible.out: 2020-03-19T21:11:21.375 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={u'ovpn': u'khiremat@hrk-thinkpad 6DWhbAZZDUo789PultFaJw f586e852c2dab5f56e3e12237dadd8f76f61d9136479329698e6f7dcd91f9034', u'name': u'khiremat'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "khiremat", "ovpn": "khiremat@hrk-thinkpad 6DWhbAZZDUo789PultFaJw f586e852c2dab5f56e3e12237dadd8f76f61d9136479329698e6f7dcd91f9034"}, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:21.401 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={u'ovpn': u'kapandya@thinkpad Uw1JHj0kPIqSd9iOJ68TNg 46badc3b5cd150bfb74daf4300c8329491000521738ba70455631aa4a47e42c0', u'name': u'kapandya'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad Uw1JHj0kPIqSd9iOJ68TNg 46badc3b5cd150bfb74daf4300c8329491000521738ba70455631aa4a47e42c0"}, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:21.475 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:11:21 +0000 (0:00:00.962) 0:00:10.079 ******** 2020-03-19T21:11:21.505 INFO:teuthology.task.ansible.out: TASK [users : Create all users without sudo access.] *************************** 2020-03-19T21:11:21.549 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:11:21 +0000 (0:00:00.073) 0:00:10.152 ******** 2020-03-19T21:11:21.574 INFO:teuthology.task.ansible.out: TASK [users : Merge managed_users and managed_admin users] ********************* 2020-03-19T21:11:21.575 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:21.614 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:11:21 +0000 (0:00:00.064) 0:00:10.217 ******** 2020-03-19T21:11:21.661 INFO:teuthology.task.ansible.out: TASK [users : Clone the keys repo] ********************************************* 2020-03-19T21:11:21.662 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:21.704 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:11:21 +0000 (0:00:00.090) 0:00:10.307 ******** 2020-03-19T21:11:21.752 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2020-03-19T21:11:21.754 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:21.794 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:11:21 +0000 (0:00:00.090) 0:00:10.398 ******** 2020-03-19T21:11:21.830 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2020-03-19T21:11:21.832 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:21.869 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:11:21 +0000 (0:00:00.074) 0:00:10.472 ******** 2020-03-19T21:11:21.905 INFO:teuthology.task.ansible.out: TASK [users : Filter the revoked_users list] *********************************** 2020-03-19T21:11:21.907 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:21.967 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:11:21 +0000 (0:00:00.098) 0:00:10.571 ******** 2020-03-19T21:11:22.003 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2020-03-19T21:11:22.005 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item=zcerza) => {"ansible_loop_var": "item", "changed": false, "item": "zcerza", "skip_reason": "Conditional result was False"} 2020-03-19T21:11:22.015 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item=aschoen) => {"ansible_loop_var": "item", "changed": false, "item": "aschoen", "skip_reason": "Conditional result was False"} 2020-03-19T21:11:22.016 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item=andrew) => {"ansible_loop_var": "item", "changed": false, "item": "andrew", "skip_reason": "Conditional result was False"} 2020-03-19T21:11:22.038 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item=sweil) => {"ansible_loop_var": "item", "changed": false, "item": "sweil", "skip_reason": "Conditional result was False"} 2020-03-19T21:11:22.039 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item=brad) => {"ansible_loop_var": "item", "changed": false, "item": "brad", "skip_reason": "Conditional result was False"} 2020-03-19T21:11:22.051 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item=kefu) => {"ansible_loop_var": "item", "changed": false, "item": "kefu", "skip_reason": "Conditional result was False"} 2020-03-19T21:11:22.063 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item=shylesh) => {"ansible_loop_var": "item", "changed": false, "item": "shylesh", "skip_reason": "Conditional result was False"} 2020-03-19T21:11:22.075 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item=trhoden) => {"ansible_loop_var": "item", "changed": false, "item": "trhoden", "skip_reason": "Conditional result was False"} 2020-03-19T21:11:22.077 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item=nishtha) => {"ansible_loop_var": "item", "changed": false, "item": "nishtha", "skip_reason": "Conditional result was False"} 2020-03-19T21:11:22.088 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item=yguang) => {"ansible_loop_var": "item", "changed": false, "item": "yguang", "skip_reason": "Conditional result was False"} 2020-03-19T21:11:22.099 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item=sdieffen) => {"ansible_loop_var": "item", "changed": false, "item": "sdieffen", "skip_reason": "Conditional result was False"} 2020-03-19T21:11:22.101 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item=brian) => {"ansible_loop_var": "item", "changed": false, "item": "brian", "skip_reason": "Conditional result was False"} 2020-03-19T21:11:22.111 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item=pmcgarry) => {"ansible_loop_var": "item", "changed": false, "item": "pmcgarry", "skip_reason": "Conditional result was False"} 2020-03-19T21:11:22.123 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item=karnan) => {"ansible_loop_var": "item", "changed": false, "item": "karnan", "skip_reason": "Conditional result was False"} 2020-03-19T21:11:22.134 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item=ryneli) => {"ansible_loop_var": "item", "changed": false, "item": "ryneli", "skip_reason": "Conditional result was False"} 2020-03-19T21:11:22.135 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item=dlambrig) => {"ansible_loop_var": "item", "changed": false, "item": "dlambrig", "skip_reason": "Conditional result was False"} 2020-03-19T21:11:22.146 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item=icolle) => {"ansible_loop_var": "item", "changed": false, "item": "icolle", "skip_reason": "Conditional result was False"} 2020-03-19T21:11:22.157 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item=soumya) => {"ansible_loop_var": "item", "changed": false, "item": "soumya", "skip_reason": "Conditional result was False"} 2020-03-19T21:11:22.168 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item=jspray) => {"ansible_loop_var": "item", "changed": false, "item": "jspray", "skip_reason": "Conditional result was False"} 2020-03-19T21:11:22.179 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item=loic) => {"ansible_loop_var": "item", "changed": false, "item": "loic", "skip_reason": "Conditional result was False"} 2020-03-19T21:11:22.180 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item=erwan) => {"ansible_loop_var": "item", "changed": false, "item": "erwan", "skip_reason": "Conditional result was False"} 2020-03-19T21:11:22.192 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item=jj) => {"ansible_loop_var": "item", "changed": false, "item": "jj", "skip_reason": "Conditional result was False"} 2020-03-19T21:11:22.203 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item=amarangone) => {"ansible_loop_var": "item", "changed": false, "item": "amarangone", "skip_reason": "Conditional result was False"} 2020-03-19T21:11:22.203 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item=oprypin) => {"ansible_loop_var": "item", "changed": false, "item": "oprypin", "skip_reason": "Conditional result was False"} 2020-03-19T21:11:22.214 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item=adamyanova) => {"ansible_loop_var": "item", "changed": false, "item": "adamyanova", "skip_reason": "Conditional result was False"} 2020-03-19T21:11:22.236 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item=sbillah) => {"ansible_loop_var": "item", "changed": false, "item": "sbillah", "skip_reason": "Conditional result was False"} 2020-03-19T21:11:22.236 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item=onyb) => {"ansible_loop_var": "item", "changed": false, "item": "onyb", "skip_reason": "Conditional result was False"} 2020-03-19T21:11:22.248 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item=jwilliamson) => {"ansible_loop_var": "item", "changed": false, "item": "jwilliamson", "skip_reason": "Conditional result was False"} 2020-03-19T21:11:22.259 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item=kmroz) => {"ansible_loop_var": "item", "changed": false, "item": "kmroz", "skip_reason": "Conditional result was False"} 2020-03-19T21:11:22.269 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item=shehbazj) => {"ansible_loop_var": "item", "changed": false, "item": "shehbazj", "skip_reason": "Conditional result was False"} 2020-03-19T21:11:22.270 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item=abhishekvrshny) => {"ansible_loop_var": "item", "changed": false, "item": "abhishekvrshny", "skip_reason": "Conditional result was False"} 2020-03-19T21:11:22.281 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item=asheplyakov) => {"ansible_loop_var": "item", "changed": false, "item": "asheplyakov", "skip_reason": "Conditional result was False"} 2020-03-19T21:11:22.282 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item=liupan) => {"ansible_loop_var": "item", "changed": false, "item": "liupan", "skip_reason": "Conditional result was False"} 2020-03-19T21:11:22.293 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item=adeza) => {"ansible_loop_var": "item", "changed": false, "item": "adeza", "skip_reason": "Conditional result was False"} 2020-03-19T21:11:22.297 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item=nlevine) => {"ansible_loop_var": "item", "changed": false, "item": "nlevine", "skip_reason": "Conditional result was False"} 2020-03-19T21:11:22.304 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item=tbrekke) => {"ansible_loop_var": "item", "changed": false, "item": "tbrekke", "skip_reason": "Conditional result was False"} 2020-03-19T21:11:22.305 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item=taco) => {"ansible_loop_var": "item", "changed": false, "item": "taco", "skip_reason": "Conditional result was False"} 2020-03-19T21:11:22.306 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item=louis) => {"ansible_loop_var": "item", "changed": false, "item": "louis", "skip_reason": "Conditional result was False"} 2020-03-19T21:11:22.317 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item=amarango) => {"ansible_loop_var": "item", "changed": false, "item": "amarango", "skip_reason": "Conditional result was False"} 2020-03-19T21:11:22.328 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item=oobe) => {"ansible_loop_var": "item", "changed": false, "item": "oobe", "skip_reason": "Conditional result was False"} 2020-03-19T21:11:22.339 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item=rturk) => {"ansible_loop_var": "item", "changed": false, "item": "rturk", "skip_reason": "Conditional result was False"} 2020-03-19T21:11:22.340 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item=fche) => {"ansible_loop_var": "item", "changed": false, "item": "fche", "skip_reason": "Conditional result was False"} 2020-03-19T21:11:22.392 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:11:22 +0000 (0:00:00.424) 0:00:10.996 ******** 2020-03-19T21:11:23.233 INFO:teuthology.task.ansible.out: TASK [users : Write /keys-repo-sha1 sentinel file] ***************************** 2020-03-19T21:11:23.235 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => {"changed": false, "checksum": "87deb5c6e571edae0503a547e464e6fdb70a7a2c", "dest": "/keys-repo-sha1", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/keys-repo-sha1", "secontext": "system_u:object_r:etc_runtime_t:s0", "size": 40, "state": "file", "uid": 0} 2020-03-19T21:11:23.301 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:11:23 +0000 (0:00:00.909) 0:00:11.905 ******** 2020-03-19T21:11:23.356 INFO:teuthology.task.ansible.out: TASK [common : Log the OS name, version and release] *************************** 2020-03-19T21:11:23.357 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => { "msg": "Host smithi170.front.sepia.ceph.com is running RedHat 8.1 (Ootpa)" } 2020-03-19T21:11:23.426 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:11:23 +0000 (0:00:00.124) 0:00:12.029 ******** 2020-03-19T21:11:23.486 INFO:teuthology.task.ansible.out: TASK [common : Including major version specific variables.] ******************** 2020-03-19T21:11:23.487 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/redhat_8.yml) => {"ansible_facts": {"nrpe_selinux_packages": ["python3-libsemanage", "python3-policycoreutils"], "rhsm_repos": ["rhel-8-for-x86_64-baseos-rpms", "rhel-8-for-x86_64-appstream-rpms"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/redhat_8.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/redhat_8.yml"} 2020-03-19T21:11:23.556 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:11:23 +0000 (0:00:00.129) 0:00:12.159 ******** 2020-03-19T21:11:23.584 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 6)] *********************** 2020-03-19T21:11:23.585 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:23.640 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:11:23 +0000 (0:00:00.084) 0:00:12.243 ******** 2020-03-19T21:11:23.920 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 7)] *********************** 2020-03-19T21:11:23.920 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => {"changed": false, "cmd": "timedatectl | grep -E \"Time ?zone\" | sed -e \"s/.*: \\(.*\\) (.*/\\1/\"", "delta": "0:00:00.071961", "end": "2020-03-19 21:11:23.895128", "rc": 0, "start": "2020-03-19 21:11:23.823167", "stderr": "", "stderr_lines": [], "stdout": "Etc/UTC", "stdout_lines": ["Etc/UTC"]} 2020-03-19T21:11:23.981 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:11:23 +0000 (0:00:00.341) 0:00:12.584 ******** 2020-03-19T21:11:24.008 INFO:teuthology.task.ansible.out: TASK [common : Reboot RHEL7 to workaround systemd bug] ************************* 2020-03-19T21:11:24.009 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:24.058 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:11:24 +0000 (0:00:00.077) 0:00:12.662 ******** 2020-03-19T21:11:24.095 INFO:teuthology.task.ansible.out: TASK [common : Wait for reboot in case of systemd workaround] ****************** skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:24.133 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:11:24 +0000 (0:00:00.074) 0:00:12.737 ******** 2020-03-19T21:11:24.161 INFO:teuthology.task.ansible.out: TASK [common : Set /etc/localtime (RHEL/CentOS 6)] ***************************** 2020-03-19T21:11:24.162 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:24.205 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:11:24 +0000 (0:00:00.072) 0:00:12.809 ******** 2020-03-19T21:11:24.230 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone (RHEL/CentOS >= 7)] **************************** 2020-03-19T21:11:24.232 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:24.271 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:11:24 +0000 (0:00:00.065) 0:00:12.875 ******** 2020-03-19T21:11:24.321 INFO:teuthology.task.ansible.out: TASK [common : Install yum on Fedora 22 and later] ***************************** 2020-03-19T21:11:24.322 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:24.369 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:11:24 +0000 (0:00:00.097) 0:00:12.972 ******** 2020-03-19T21:11:24.428 INFO:teuthology.task.ansible.out: TASK [common : Set entitlements_path] ****************************************** 2020-03-19T21:11:24.429 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => {"ansible_facts": {"entitlements_path": "/etc/ansible/secrets/entitlements.yml"}, "changed": false} 2020-03-19T21:11:24.479 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:11:24 +0000 (0:00:00.109) 0:00:13.082 ******** 2020-03-19T21:11:24.535 INFO:teuthology.task.ansible.out: TASK [common : Include Red Hat encrypted variables.] *************************** 2020-03-19T21:11:24.536 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item=None) => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} ok: [smithi170.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-03-19T21:11:24.581 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:11:24 +0000 (0:00:00.102) 0:00:13.185 ******** 2020-03-19T21:11:24.617 INFO:teuthology.task.ansible.out: TASK [common : Set have_entitlements] ****************************************** 2020-03-19T21:11:24.618 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => {"ansible_facts": {"have_entitlements": true}, "changed": false} 2020-03-19T21:11:24.655 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:11:24 +0000 (0:00:00.074) 0:00:13.259 ******** 2020-03-19T21:11:25.896 INFO:teuthology.task.ansible.out: TASK [common : Install CA Cert from Satellite Server] ************************** 2020-03-19T21:11:25.896 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": ["Installed /root/.ansible/tmp/ansible-moduletmp-1584652285.7791955-er8ehkjy/katello-ca-consumer-latest.noarch9rf1r5av.rpm"]} 2020-03-19T21:11:25.943 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:11:25 +0000 (0:00:01.287) 0:00:14.546 ******** 2020-03-19T21:11:26.862 INFO:teuthology.task.ansible.out: TASK [common : Determine if node is registered with subscription-manager.] ***** 2020-03-19T21:11:26.864 INFO:teuthology.task.ansible.out:fatal: [smithi170.front.sepia.ceph.com]: FAILED! => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} ...ignoring 2020-03-19T21:11:26.927 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:11:26 +0000 (0:00:00.984) 0:00:15.531 ******** 2020-03-19T21:11:26.965 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we're already registered] **************** 2020-03-19T21:11:26.966 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => {"ansible_facts": {"rhsm_registered": false}, "changed": false} 2020-03-19T21:11:27.013 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:11:27 +0000 (0:00:00.085) 0:00:15.616 ******** 2020-03-19T21:11:36.644 INFO:teuthology.task.ansible.out: TASK [common : Register with subscription-manager.] **************************** changed: [smithi170.front.sepia.ceph.com] => {"attempts": 1, "censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": true} 2020-03-19T21:11:36.682 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:11:36 +0000 (0:00:09.669) 0:00:25.286 ******** 2020-03-19T21:11:36.754 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we just registered] ********************** ok: [smithi170.front.sepia.ceph.com] => {"ansible_facts": {"rhsm_registered": true}, "changed": false} 2020-03-19T21:11:36.848 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:11:36 +0000 (0:00:00.165) 0:00:25.451 ******** 2020-03-19T21:11:37.791 INFO:teuthology.task.ansible.out: TASK [common : List CDN releases available to system] ************************** 2020-03-19T21:11:37.792 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => {"changed": false, "cmd": "subscription-manager release --list | grep -E '[0-9]'", "delta": "0:00:00.662436", "end": "2020-03-19 21:11:37.751884", "failed_when_result": false, "rc": 0, "start": "2020-03-19 21:11:37.089448", "stderr": "", "stderr_lines": [], "stdout": "7.4\n7.5\n7.6\n7.7\n7Server\n8\n8.0\n8.1", "stdout_lines": ["7.4", "7.5", "7.6", "7.7", "7Server", "8", "8.0", "8.1"]} 2020-03-19T21:11:37.833 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:11:37 +0000 (0:00:00.985) 0:00:26.437 ******** 2020-03-19T21:11:37.861 INFO:teuthology.task.ansible.out: TASK [common : Unregister Beta/Alpha/RC system with subscription-manager] ****** 2020-03-19T21:11:37.863 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:37.902 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:11:37 +0000 (0:00:00.068) 0:00:26.505 ******** 2020-03-19T21:11:37.943 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered to false if Beta/Alpha/RC release] ********** 2020-03-19T21:11:37.945 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:37.980 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:11:37 +0000 (0:00:00.078) 0:00:26.584 ******** 2020-03-19T21:11:38.006 INFO:teuthology.task.ansible.out: TASK [common : Configure internal beta repos in /etc/yum.repos.d/] ************* 2020-03-19T21:11:38.008 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-03-19T21:11:38.077 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:11:38 +0000 (0:00:00.096) 0:00:26.680 ******** 2020-03-19T21:11:38.124 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:38.194 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:11:38 +0000 (0:00:00.117) 0:00:26.798 ******** 2020-03-19T21:11:47.873 INFO:teuthology.task.ansible.out: TASK [common : Get list of enabled RHSM repos] ********************************* 2020-03-19T21:11:47.873 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => {"changed": false, "cmd": "subscription-manager repos --list | grep -B4 'Enabled:.*1' | grep 'Repo ID:' | sed -e 's/Repo ID:\\s*\\(.*\\)/\\1/' | sort", "delta": "0:00:09.429372", "end": "2020-03-19 21:11:47.846803", "rc": 0, "start": "2020-03-19 21:11:38.417431", "stderr": "", "stderr_lines": [], "stdout": "rhel-8-for-x86_64-appstream-rpms\nrhel-8-for-x86_64-baseos-rpms", "stdout_lines": ["rhel-8-for-x86_64-appstream-rpms", "rhel-8-for-x86_64-baseos-rpms"]} 2020-03-19T21:11:47.909 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:11:47 +0000 (0:00:09.715) 0:00:36.513 ******** 2020-03-19T21:11:47.951 INFO:teuthology.task.ansible.out: TASK [common : Store list of enabled repos] ************************************ 2020-03-19T21:11:47.952 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => {"ansible_facts": {"repo_list": ["rhel-8-for-x86_64-appstream-rpms", "rhel-8-for-x86_64-baseos-rpms"]}, "changed": false} 2020-03-19T21:11:48.014 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:11:48 +0000 (0:00:00.104) 0:00:36.618 ******** 2020-03-19T21:11:48.077 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos false if entitlements are missing or if we unregistered] *** 2020-03-19T21:11:48.078 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:11:48.133 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:11:48 +0000 (0:00:00.115) 0:00:36.733 ******** 2020-03-19T21:11:48.190 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if rhsm_repos differs from repo_list] **** 2020-03-19T21:11:48.191 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => {"ansible_facts": {"replace_repos": false}, "changed": false} 2020-03-19T21:11:48.233 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:11:48 +0000 (0:00:00.103) 0:00:36.837 ******** 2020-03-19T21:11:48.290 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if newly-subscribed] ********************* 2020-03-19T21:11:48.292 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => {"ansible_facts": {"replace_repos": true}, "changed": false} 2020-03-19T21:11:48.377 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:11:48 +0000 (0:00:00.144) 0:00:36.981 ******** 2020-03-19T21:11:53.931 INFO:teuthology.task.ansible.out: TASK [common : Disable all rhsm repos] ***************************************** 2020-03-19T21:11:53.932 INFO:teuthology.task.ansible.out:changed: [smithi170.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": true} 2020-03-19T21:11:53.988 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:11:53 +0000 (0:00:05.610) 0:00:42.592 ******** 2020-03-19T21:12:00.538 INFO:teuthology.task.ansible.out: TASK [common : Enable necessary rhsm repos] ************************************ 2020-03-19T21:12:00.540 INFO:teuthology.task.ansible.out:changed: [smithi170.front.sepia.ceph.com] => (item=rhel-8-for-x86_64-baseos-rpms) => {"ansible_loop_var": "item", "changed": true, "cmd": ["subscription-manager", "repos", "--enable", "rhel-8-for-x86_64-baseos-rpms"], "delta": "0:00:06.315688", "end": "2020-03-19 21:12:00.507855", "item": "rhel-8-for-x86_64-baseos-rpms", "rc": 0, "start": "2020-03-19 21:11:54.192167", "stderr": "", "stderr_lines": [], "stdout": "Internal Server Error): Required lock is already taken by other running tasks.\nPlease inspect their state, fix their errors and resume them.\n\nRequired lock: update\nConflicts with tasks:\n https://satellite.front.sepia.ceph.com/foreman_tasks/tasks/f3782032 3e38 4c74 952c dbf8a5fd4549\n\nRepository 'rhel-8-for-x86_64-baseos-rpms' is enabled for this system.", "stdout_lines": ["Internal Server Error): Required lock is already taken by other running tasks.", "Please inspect their state, fix their errors and resume them.", "", "Required lock: update", "Conflicts with tasks:", " https://satellite.front.sepia.ceph.com/foreman_tasks/tasks/f3782032 3e38 4c74 952c dbf8a5fd4549", "", "Repository 'rhel-8-for-x86_64-baseos-rpms' is enabled for this system."]} 2020-03-19T21:12:06.407 INFO:teuthology.task.ansible.out:changed: [smithi170.front.sepia.ceph.com] => (item=rhel-8-for-x86_64-appstream-rpms) => {"ansible_loop_var": "item", "changed": true, "cmd": ["subscription-manager", "repos", "--enable", "rhel-8-for-x86_64-appstream-rpms"], "delta": "0:00:05.679902", "end": "2020-03-19 21:12:06.386625", "item": "rhel-8-for-x86_64-appstream-rpms", "rc": 0, "start": "2020-03-19 21:12:00.706723", "stderr": "", "stderr_lines": [], "stdout": "Repository 'rhel-8-for-x86_64-appstream-rpms' is enabled for this system.", "stdout_lines": ["Repository 'rhel-8-for-x86_64-appstream-rpms' is enabled for this system."]} 2020-03-19T21:12:06.445 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:12:06 +0000 (0:00:12.457) 0:00:55.049 ******** 2020-03-19T21:12:06.667 INFO:teuthology.task.ansible.out: TASK [common : Remove old apt-mirror repository definition.] ******************* 2020-03-19T21:12:06.668 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => {"changed": false, "path": "/etc/yum.repos.d/cd.repo", "state": "absent"} 2020-03-19T21:12:06.703 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:12:06 +0000 (0:00:00.258) 0:00:55.307 ******** 2020-03-19T21:12:06.904 INFO:teuthology.task.ansible.out: TASK [common : Increase the yum timeout.] ************************************** 2020-03-19T21:12:06.905 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-03-19T21:12:06.944 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:12:06 +0000 (0:00:00.240) 0:00:55.548 ******** 2020-03-19T21:12:07.483 INFO:teuthology.task.ansible.out: TASK [common : Configure epel repos in /etc/yum.repos.d/] ********************** 2020-03-19T21:12:07.483 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'value': {u'gpgcheck': 0, u'mirrorlist': u'file:///etc/yum.repos.d/epel-mirrorlist', u'enabled': u'1', u'name': u'Extra Packages for Enterprise Linux', u'failovermethod': u'priority'}, 'key': u'epel'}) => {"ansible_loop_var": "item", "changed": false, "checksum": "86c0d58bde7ad4cfaafe592d2fefd92c920c42ca", "dest": "/etc/yum.repos.d/epel.repo", "gid": 0, "group": "root", "item": {"key": "epel", "value": {"enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 236, "state": "file", "uid": 0} 2020-03-19T21:12:07.940 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'value': {u'gpgcheck': 0, u'mirrorlist': u'file:///etc/yum.repos.d/epel-testing-mirrorlist', u'enabled': 0, u'name': u'Extra Packages for Enterprise Linux - Testing', u'failovermethod': u'priority'}, 'key': u'epel-testing'}) => {"ansible_loop_var": "item", "changed": false, "checksum": "a1bad5509ed35b895c2061841236b1cfe10619ae", "dest": "/etc/yum.repos.d/epel-testing.repo", "gid": 0, "group": "root", "item": {"key": "epel-testing", "value": {"enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel-testing.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 262, "state": "file", "uid": 0} 2020-03-19T21:12:07.982 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:12:07 +0000 (0:00:01.037) 0:00:56.585 ******** 2020-03-19T21:12:08.511 INFO:teuthology.task.ansible.out: TASK [common : Configure local epel mirrorlists] ******************************* 2020-03-19T21:12:08.513 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item=epel-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "checksum": "3c9ba7bacd1e30bed014cefd1fb4eab0f0f8fa10", "dest": "/etc/yum.repos.d/epel-mirrorlist", "gid": 0, "group": "root", "item": "epel-mirrorlist", "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel-mirrorlist", "secontext": "system_u:object_r:system_conf_t:s0", "size": 484, "state": "file", "uid": 0} 2020-03-19T21:12:09.007 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item=epel-testing-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "checksum": "d542c3e2c30a0d3eb7d454775342c302c0ded15b", "dest": "/etc/yum.repos.d/epel-testing-mirrorlist", "gid": 0, "group": "root", "item": "epel-testing-mirrorlist", "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel-testing-mirrorlist", "secontext": "system_u:object_r:system_conf_t:s0", "size": 540, "state": "file", "uid": 0} 2020-03-19T21:12:09.056 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:12:09 +0000 (0:00:01.074) 0:00:57.659 ******** 2020-03-19T21:12:09.102 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ 2020-03-19T21:12:09.104 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:12:09.150 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:12:09 +0000 (0:00:00.083) 0:00:57.743 ******** 2020-03-19T21:12:09.164 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** 2020-03-19T21:12:09.166 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:12:09.211 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:12:09 +0000 (0:00:00.072) 0:00:57.815 ******** 2020-03-19T21:12:09.247 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone in /etc/timezone] ****************************** 2020-03-19T21:12:09.248 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:12:09.300 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:12:09 +0000 (0:00:00.088) 0:00:57.904 ******** 2020-03-19T21:12:09.328 INFO:teuthology.task.ansible.out: TASK [common : Inform the OS of the new timezone] ****************************** 2020-03-19T21:12:09.329 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:12:09.386 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:12:09 +0000 (0:00:00.085) 0:00:57.989 ******** 2020-03-19T21:12:09.415 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** 2020-03-19T21:12:09.416 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:12:09.471 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:12:09 +0000 (0:00:00.084) 0:00:58.074 ******** 2020-03-19T21:12:09.498 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone] *********************************************** 2020-03-19T21:12:09.500 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:12:09.553 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:12:09 +0000 (0:00:00.082) 0:00:58.156 ******** 2020-03-19T21:12:09.579 INFO:teuthology.task.ansible.out: TASK [common : Add base OpenSUSE Leap repo] ************************************ 2020-03-19T21:12:09.580 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:12:09.618 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:12:09 +0000 (0:00:00.065) 0:00:58.222 ******** 2020-03-19T21:12:09.644 INFO:teuthology.task.ansible.out: TASK [common : Add updates OpenSUSE Leap repo] ********************************* 2020-03-19T21:12:09.646 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:12:09.685 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:12:09 +0000 (0:00:00.066) 0:00:58.288 ******** 2020-03-19T21:12:09.710 INFO:teuthology.task.ansible.out: TASK [common : Refresh repos] ************************************************** 2020-03-19T21:12:09.712 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:12:09.745 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:12:09 +0000 (0:00:00.060) 0:00:58.349 ******** 2020-03-19T21:12:10.521 INFO:teuthology.task.ansible.out: TASK [common : Set the hardware clock] ***************************************** 2020-03-19T21:12:10.523 INFO:teuthology.task.ansible.out:changed: [smithi170.front.sepia.ceph.com] => {"changed": true, "cmd": ["hwclock", "--systohc"], "delta": "0:00:00.578878", "end": "2020-03-19 21:12:10.500331", "rc": 0, "start": "2020-03-19 21:12:09.921453", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-03-19T21:12:10.564 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:12:10 +0000 (0:00:00.815) 0:00:59.165 ******** 2020-03-19T21:12:12.692 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (RedHat)] ***************************** 2020-03-19T21:12:12.694 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": ["Installed: krb5-workstation"]} 2020-03-19T21:12:12.733 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:12:12 +0000 (0:00:02.172) 0:01:01.337 ******** 2020-03-19T21:12:12.759 INFO:teuthology.task.ansible.out: TASK [common : Clean apt cache] ************************************************ 2020-03-19T21:12:12.760 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:12:12.798 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:12:12 +0000 (0:00:00.064) 0:01:01.402 ******** 2020-03-19T21:12:12.825 INFO:teuthology.task.ansible.out: TASK [common : Update apt cache] *********************************************** 2020-03-19T21:12:12.826 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:12:12.860 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:12:12 +0000 (0:00:00.061) 0:01:01.463 ******** 2020-03-19T21:12:12.885 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (Debian)] ***************************** 2020-03-19T21:12:12.886 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:12:12.927 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:12:12 +0000 (0:00:00.064) 0:01:01.528 ******** 2020-03-19T21:12:12.950 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (OpenSUSE Leap)] ********************** 2020-03-19T21:12:12.951 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:12:12.986 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:12:12 +0000 (0:00:00.061) 0:01:01.589 ******** 2020-03-19T21:12:13.475 INFO:teuthology.task.ansible.out: TASK [common : Add krb5 config file] ******************************************* 2020-03-19T21:12:13.477 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => {"changed": false, "checksum": "b3c1a33d19a4e448c29374429629dff57657b576", "dest": "/etc/krb5.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/krb5.conf", "secontext": "system_u:object_r:krb5_conf_t:s0", "size": 131, "state": "file", "uid": 0} 2020-03-19T21:12:13.523 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:12:13 +0000 (0:00:00.537) 0:01:02.127 ******** 2020-03-19T21:12:15.407 INFO:teuthology.task.ansible.out: TASK [common : Make sure smartmontools is installed] *************************** 2020-03-19T21:12:15.409 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": ["Installed: smartmontools"]} 2020-03-19T21:12:15.452 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:12:15 +0000 (0:00:01.928) 0:01:04.056 ******** 2020-03-19T21:12:15.956 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** 2020-03-19T21:12:15.957 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item=megacli) => {"ansible_loop_var": "item", "changed": false, "checksum": "cb3ef2aa6a3e3002b234f166e739a39b1f3a1bde", "dest": "/usr/sbin/megacli", "gid": 0, "group": "root", "item": "megacli", "mode": "0755", "owner": "root", "path": "/usr/sbin/megacli", "secontext": "system_u:object_r:bin_t:s0", "size": 2716736, "state": "file", "uid": 0} 2020-03-19T21:12:16.406 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item=cli64) => {"ansible_loop_var": "item", "changed": false, "checksum": "a3b8773b14e8594ffbd9fc156c09436eb0184bc7", "dest": "/usr/sbin/cli64", "gid": 0, "group": "root", "item": "cli64", "mode": "0755", "owner": "root", "path": "/usr/sbin/cli64", "secontext": "system_u:object_r:bin_t:s0", "size": 1952038, "state": "file", "uid": 0} 2020-03-19T21:12:16.842 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item=nvme) => {"ansible_loop_var": "item", "changed": false, "checksum": "98735dcacd6fec55dd92b67281b478603b7955b1", "dest": "/usr/sbin/nvme", "gid": 0, "group": "root", "item": "nvme", "mode": "0755", "owner": "root", "path": "/usr/sbin/nvme", "secontext": "system_u:object_r:bin_t:s0", "size": 580432, "state": "file", "uid": 0} 2020-03-19T21:12:16.888 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:12:16 +0000 (0:00:01.435) 0:01:05.491 ******** 2020-03-19T21:12:17.084 INFO:teuthology.task.ansible.out: TASK [common : Create /usr/libexec.] ******************************************* 2020-03-19T21:12:17.085 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/libexec", "secontext": "system_u:object_r:bin_t:s0", "size": 4096, "state": "directory", "uid": 0} 2020-03-19T21:12:17.128 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:12:17 +0000 (0:00:00.239) 0:01:05.731 ******** 2020-03-19T21:12:17.613 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2020-03-19T21:12:17.613 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item=smart.sh) => {"ansible_loop_var": "item", "changed": false, "checksum": "9a447e184389aadb9aaf1e77f30f0a7f8b716c4e", "dest": "/usr/libexec/smart.sh", "gid": 0, "group": "root", "item": "smart.sh", "mode": "0755", "owner": "root", "path": "/usr/libexec/smart.sh", "secontext": "system_u:object_r:bin_t:s0", "size": 12725, "state": "file", "uid": 0} 2020-03-19T21:12:18.038 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item=raid.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9d9a89805d8a9e4c7281934e887da31f7ac464fb", "dest": "/usr/libexec/raid.pl", "gid": 0, "group": "root", "item": "raid.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/raid.pl", "secontext": "system_u:object_r:bin_t:s0", "size": 8602, "state": "file", "uid": 0} 2020-03-19T21:12:18.485 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item=diskusage.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9c74425885a83f23bc37817de46979ca5173ccb7", "dest": "/usr/libexec/diskusage.pl", "gid": 0, "group": "root", "item": "diskusage.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/diskusage.pl", "secontext": "system_u:object_r:bin_t:s0", "size": 3605, "state": "file", "uid": 0} 2020-03-19T21:12:18.525 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:12:18 +0000 (0:00:01.397) 0:01:07.129 ******** 2020-03-19T21:12:18.550 INFO:teuthology.task.ansible.out: TASK [common : Include apt_system vars] **************************************** skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:12:18.606 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:12:18 +0000 (0:00:00.081) 0:01:07.210 ******** 2020-03-19T21:12:18.608 INFO:teuthology.task.ansible.out: 2020-03-19T21:12:18.644 INFO:teuthology.task.ansible.out: TASK [common : Include yum_systems vars] *************************************** 2020-03-19T21:12:18.644 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => {"ansible_facts": {"nagios_plugins_directory": "/usr/lib64/nagios/plugins", "nrpe_group": "nrpe", "nrpe_packages": ["nagios-common", "nrpe", "nagios-plugins", "nagios-plugins-load"], "nrpe_service_name": "nrpe", "nrpe_user": "nrpe"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/yum_systems.yml"], "changed": false} 2020-03-19T21:12:18.688 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:12:18 +0000 (0:00:00.081) 0:01:07.292 ******** 2020-03-19T21:12:18.890 INFO:teuthology.task.ansible.out: TASK [common : Check for epel] ************************************************* 2020-03-19T21:12:18.892 INFO:teuthology.task.ansible.out:changed: [smithi170.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": true} 2020-03-19T21:12:18.936 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:12:18 +0000 (0:00:00.247) 0:01:07.539 ******** 2020-03-19T21:12:18.971 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe without epel] ************************************** 2020-03-19T21:12:18.973 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-03-19T21:12:18.982 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-03-19T21:12:18.993 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-03-19T21:12:19.004 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-03-19T21:12:19.041 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:12:19 +0000 (0:00:00.105) 0:01:07.645 ******** 2020-03-19T21:12:21.000 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (RHEL/CentOS)] ************ ok: [smithi170.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": ["Installed: nagios-common", "Installed: nrpe", "Installed: nagios-plugins", "Installed: nagios-plugins-load"]} 2020-03-19T21:12:21.037 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:12:21 +0000 (0:00:01.995) 0:01:09.640 ******** 2020-03-19T21:12:21.062 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (Ubuntu/Fedora)] ********** 2020-03-19T21:12:21.063 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:12:21.100 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:12:21 +0000 (0:00:00.063) 0:01:09.703 ******** 2020-03-19T21:12:21.586 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios sudoers.d for raid utilities.] ******************** 2020-03-19T21:12:21.588 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => {"changed": false, "checksum": "b423b985ccbb69b65a711d372645a5f704b30e8b", "dest": "/etc/sudoers.d/90-nagios", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers.d/90-nagios", "secontext": "system_u:object_r:etc_t:s0", "size": 178, "state": "file", "uid": 0} 2020-03-19T21:12:21.624 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:12:21 +0000 (0:00:00.524) 0:01:10.228 ******** 2020-03-19T21:12:22.108 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios check_mem script] ********************************* 2020-03-19T21:12:22.108 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => {"changed": false, "checksum": "39df629b10ded370443e2e4c84d690332c95104d", "dest": "/usr/lib64/nagios/plugins/check_mem.sh", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib64/nagios/plugins/check_mem.sh", "secontext": "system_u:object_r:nagios_unconfined_plugin_exec_t:s0", "size": 1964, "state": "file", "uid": 0} 2020-03-19T21:12:22.162 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:12:22 +0000 (0:00:00.537) 0:01:10.765 ******** 2020-03-19T21:12:22.212 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (Ubuntu)] ************************ 2020-03-19T21:12:22.214 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:12:22.251 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:12:22 +0000 (0:00:00.088) 0:01:10.854 ******** 2020-03-19T21:12:22.469 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (RHEL/CentOS)] ******************* 2020-03-19T21:12:22.469 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-03-19T21:12:22.511 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:12:22 +0000 (0:00:00.259) 0:01:11.114 ******** 2020-03-19T21:12:22.775 INFO:teuthology.task.ansible.out: TASK [common : Check firewalld status] ***************************************** 2020-03-19T21:12:22.776 INFO:teuthology.task.ansible.out:fatal: [smithi170.front.sepia.ceph.com]: FAILED! => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": true} ...ignoring 2020-03-19T21:12:22.812 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:12:22 +0000 (0:00:00.300) 0:01:11.415 ******** 2020-03-19T21:12:22.837 INFO:teuthology.task.ansible.out: TASK [common : Open nrpe port if firewalld enabled] **************************** 2020-03-19T21:12:22.838 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:12:22.873 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:12:22 +0000 (0:00:00.061) 0:01:11.477 ******** 2020-03-19T21:12:23.355 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios nrpe config.] ************************************* 2020-03-19T21:12:23.357 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => {"changed": false, "checksum": "333141ba5e85aea94f51511a52d9b115d2e0112f", "dest": "/etc/nagios/nrpe.cfg", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/nagios/nrpe.cfg", "secontext": "system_u:object_r:nrpe_etc_t:s0", "size": 1174, "state": "file", "uid": 0} 2020-03-19T21:12:23.392 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:12:23 +0000 (0:00:00.518) 0:01:11.996 ******** 2020-03-19T21:12:24.183 INFO:teuthology.task.ansible.out: TASK [common : Make sure nagios nrpe service is running.] ********************** 2020-03-19T21:12:24.185 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "nrpe", "state": "started", "status": {"ActiveEnterTimestamp": "Thu 2020-03-19 21:10:20 UTC", "ActiveEnterTimestampMonotonic": "15051196", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "var-run.mount tmp.mount systemd-journald.socket local-fs.target nss-lookup.target -.mount system.slice network.target time-sync.target basic.target systemd-tmpfiles-setup.service sysinit.target", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Thu 2020-03-19 21:10:20 UTC", "AssertTimestampMonotonic": "15050416", "Before": "getty@tty1.service plymouth-quit.service xdm.service shutdown.target multi-user.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Thu 2020-03-19 21:10:20 UTC", "ConditionTimestampMonotonic": "15050415", "ConfigurationDirectoryMode": "0755", "Conflicts": "nrpe.socket shutdown.target", "ControlGroup": "/system.slice/nrpe.service", "ControlPID": "0", "D 2020-03-19T21:12:24.186 INFO:teuthology.task.ansible.out:efaultDependencies": "yes", "Delegate": "no", "Description": "Nagios Remote Plugin Executor", "DevicePolicy": "auto", "Documentation": "http://www.nagios.org/documentation", "DynamicUser": "no", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "2417", "ExecMainStartTimestamp": "Thu 2020-03-19 21:10:20 UTC", "ExecMainStartTimestampMonotonic": "15051168", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/nrpe ; argv[]=/usr/sbin/nrpe -c /etc/nagios/nrpe.cfg -f ; ignore_errors=no ; start_time=[Thu 2020-03-19 21:10:20 UTC] ; stop_time=[n/a] ; pid=2417 ; code=(null) ; status=0/0 }", "ExecStopPost": "{ path=/bin/rm ; argv[]=/bin/rm -f /var/run/nrpe/nrpe.pid ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/nrpe.service", "GID": "986", "Group": "nrpe", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "nrpe.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Thu 2020-03-19 21:10:20 UTC", "InactiveExitTimestampMonotonic": "15051196", "InvocationID": "1dcce6aff7e24d3c84b3cce5f127f031", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "infinity", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "Limi 2020-03-19T21:12:24.188 INFO:teuthology.task.ansible.out:tDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "16777216", "LimitMEMLOCKSoft": "16777216", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "127442", "LimitNPROCSoft": "127442", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "127442", "LimitSIGPENDINGSoft": "127442", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "2417", "MemoryAccounting": "yes", "MemoryCurrent": "1073152", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "nrpe.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "-500", "OnFailureJobMode": "replace", "PIDFile": "/var/run/nrpe/nrpe.pid", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateMounts": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "sysinit.target -.mount system.slice", "RequiresMountsFor": "/var/tmp /run/nrpe", "Restart": "on-abort", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectory": "n 2020-03-19T21:12:24.189 INFO:teuthology.task.ansible.out:rpe", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Thu 2020-03-19 21:10:20 UTC", "StateChangeTimestampMonotonic": "15051196", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "1", "TasksMax": "26213", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min", "TimerSlackNSec": "50000", "Transient": "no", "Type": "simple", "UID": "990", "UMask": "0022", "UnitFilePreset": "disabled", "UnitFileState": "enabled", "User": "nrpe", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Thu 2020-03-19 21:10:20 UTC", "WatchdogTimestampMonotonic": "15051195", "WatchdogUSec": "0"}} 2020-03-19T21:12:24.223 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:12:24 +0000 (0:00:00.830) 0:01:12.826 ******** 2020-03-19T21:12:24.415 INFO:teuthology.task.ansible.out: TASK [common : Get SELinux status] ********************************************* 2020-03-19T21:12:24.417 INFO:teuthology.task.ansible.out:changed: [smithi170.front.sepia.ceph.com] => {"changed": true, "cmd": ["getenforce"], "delta": "0:00:00.001736", "end": "2020-03-19 21:12:24.393447", "rc": 0, "start": "2020-03-19 21:12:24.391711", "stderr": "", "stderr_lines": [], "stdout": "Permissive", "stdout_lines": ["Permissive"]} 2020-03-19T21:12:24.451 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:12:24 +0000 (0:00:00.228) 0:01:13.054 ******** 2020-03-19T21:12:26.365 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Install SELinux dependencies] **************************** 2020-03-19T21:12:26.367 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": ["Installed: python3-libsemanage", "Installed: python3-policycoreutils"]} 2020-03-19T21:12:26.446 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:12:26 +0000 (0:00:01.995) 0:01:15.050 ******** 2020-03-19T21:12:28.372 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Ensure SELinux policy is up to date] ********************* 2020-03-19T21:12:28.372 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": ["Installed: selinux-policy-targeted"]} 2020-03-19T21:12:28.411 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:12:28 +0000 (0:00:01.964) 0:01:17.014 ******** 2020-03-19T21:12:29.859 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Set SELinux boolean nagios_run_sudo true] **************** 2020-03-19T21:12:29.860 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => {"changed": false, "name": "nagios_run_sudo", "persistent": true, "state": true} 2020-03-19T21:12:29.901 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:12:29 +0000 (0:00:01.489) 0:01:18.504 ******** 2020-03-19T21:12:46.320 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove SELinux policy package] *************************** 2020-03-19T21:12:46.321 INFO:teuthology.task.ansible.out:changed: [smithi170.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule", "-r", "nrpe"], "delta": "0:00:16.216444", "end": "2020-03-19 21:12:46.292950", "failed_when_result": false, "rc": 0, "start": "2020-03-19 21:12:30.076506", "stderr": "", "stderr_lines": [], "stdout": "libsemanage.semanage_direct_remove_key: Removing last nrpe module (no other nrpe module exists at another priority).", "stdout_lines": ["libsemanage.semanage_direct_remove_key: Removing last nrpe module (no other nrpe module exists at another priority)."]} 2020-03-19T21:12:46.355 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:12:46 +0000 (0:00:16.453) 0:01:34.958 ******** 2020-03-19T21:12:46.853 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Copy SELinux type enforcement file] ********************** 2020-03-19T21:12:46.854 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => {"changed": false, "checksum": "99d9b2ea41fb2d6613898129718064833d2de428", "dest": "/tmp/nrpe.te", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/tmp/nrpe.te", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 934, "state": "file", "uid": 0} 2020-03-19T21:12:46.890 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:12:46 +0000 (0:00:00.535) 0:01:35.493 ******** 2020-03-19T21:12:47.123 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Compile SELinux module file] ***************************** 2020-03-19T21:12:47.124 INFO:teuthology.task.ansible.out:changed: [smithi170.front.sepia.ceph.com] => {"changed": true, "cmd": ["checkmodule", "-M", "-m", "-o", "/tmp/nrpe.mod", "/tmp/nrpe.te"], "delta": "0:00:00.025584", "end": "2020-03-19 21:12:47.097987", "rc": 0, "start": "2020-03-19 21:12:47.072403", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-03-19T21:12:47.160 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:12:47 +0000 (0:00:00.270) 0:01:35.764 ******** 2020-03-19T21:12:47.352 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Build SELinux policy package] **************************** 2020-03-19T21:12:47.353 INFO:teuthology.task.ansible.out:changed: [smithi170.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule_package", "-o", "/tmp/nrpe.pp", "-m", "/tmp/nrpe.mod"], "delta": "0:00:00.002135", "end": "2020-03-19 21:12:47.330454", "rc": 0, "start": "2020-03-19 21:12:47.328319", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-03-19T21:12:47.389 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:12:47 +0000 (0:00:00.228) 0:01:35.993 ******** 2020-03-19T21:13:03.966 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Load SELinux policy package] ***************************** 2020-03-19T21:13:03.967 INFO:teuthology.task.ansible.out:changed: [smithi170.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule", "-i", "/tmp/nrpe.pp"], "delta": "0:00:16.374383", "end": "2020-03-19 21:13:03.935115", "rc": 0, "start": "2020-03-19 21:12:47.560732", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-03-19T21:13:04.016 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:13:04 +0000 (0:00:16.626) 0:01:52.619 ******** 2020-03-19T21:13:04.245 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove temporary files] ********************************** 2020-03-19T21:13:04.246 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => {"changed": false, "path": "/tmp/nrpe.*", "state": "absent"} 2020-03-19T21:13:04.309 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:13:04 +0000 (0:00:00.293) 0:01:52.913 ******** 2020-03-19T21:13:04.350 INFO:teuthology.task.ansible.out: TASK [testnode : Include package type specific vars.] ************************** 2020-03-19T21:13:04.351 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => {"ansible_facts": {"ceph_dependency_packages_to_remove": ["boost-random", "boost-program-options", "leveldb", "xmlstarlet", "python-jinja2", "python-ceph", "python-flask", "python-requests", "boost-random", "python-urllib3", "python-babel", "hdparm", "python-markupsafe", "python-werkzeug", "python-itsdangerous"], "ceph_packages_to_remove": ["ceph", "ceph-base", "ceph-selinux", "ceph-common", "ceph-debuginfo", "ceph-release", "libcephfs1", "ceph-radosgw", "python-ceph", "python-rados", "python-rbd", "python-cephfs", "librbd1", "librados2", "mod_fastcgi", "iozone"], "ntp_service_name": "ntpd", "packages_to_remove": ["device-mapper-multipath"], "pip_packages_to_install": ["remoto>=0.0.35"], "ssh_service_name": "sshd"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/dnf_systems.yml"], "changed": false} 2020-03-19T21:13:04.410 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:13:04 +0000 (0:00:00.100) 0:01:53.013 ******** 2020-03-19T21:13:04.468 INFO:teuthology.task.ansible.out: TASK [testnode : Including distro specific variables.] ************************* 2020-03-19T21:13:04.470 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2020-03-19T21:13:04.509 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:13:04 +0000 (0:00:00.099) 0:01:53.113 ******** 2020-03-19T21:13:04.579 INFO:teuthology.task.ansible.out: TASK [testnode : Including major version specific variables.] ****************** 2020-03-19T21:13:04.583 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/redhat_8.yml) => {"ansible_facts": {"common_yum_repos": {"lab-extras": {"baseurl": "http://{{ mirror_host }}/lab-extras/8/", "enabled": 1, "gpgcheck": 0, "name": "lab-extras"}}, "copr_repos": ["ktdreyer/ceph-el8"], "epel_packages": ["dbench"], "nfs_service": "nfs-server", "ntp_service_name": "chronyd", "packages": ["@core", "@base", "dnf-utils", "git-all", "sysstat", "libedit", "boost-thread", "xfsprogs", "gdisk", "parted", "libgcrypt", "fuse-libs", "openssl", "libuuid", "attr", "ant", "lsof", "gettext", "bc", "xfsdump", "blktrace", "usbredir", "podman", "podman-docker", "libev-devel", "valgrind", "nfs-utils", "ncurses-devel", "gcc", "git", "python3-nose", "python3-virtualenv", "genisoimage", "qemu-img", "qemu-kvm-core", "qemu-kvm-block-rbd", "libacl-devel", "autoconf", "gdb"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/redhat_8.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/redhat_8.yml"} 2020-03-19T21:13:04.643 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:13:04 +0000 (0:00:00.131) 0:01:53.244 ******** 2020-03-19T21:13:04.678 INFO:teuthology.task.ansible.out: TASK [testnode : Including version specific variables.] ************************ 2020-03-19T21:13:04.680 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2020-03-19T21:13:04.742 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:13:04 +0000 (0:00:00.101) 0:01:53.346 ******** 2020-03-19T21:13:04.945 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the sudo group exists.] ******************************** 2020-03-19T21:13:04.947 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => {"changed": false, "gid": 1000, "name": "sudo", "state": "present", "system": false} 2020-03-19T21:13:04.981 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:13:04 +0000 (0:00:00.239) 0:01:53.585 ******** 2020-03-19T21:13:05.182 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the teuthology_user group exists.] ********************* 2020-03-19T21:13:05.183 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => {"changed": false, "gid": 1211, "name": "ubuntu", "state": "present", "system": false} 2020-03-19T21:13:05.218 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:13:05 +0000 (0:00:00.236) 0:01:53.821 ******** 2020-03-19T21:13:05.253 INFO:teuthology.task.ansible.out: TASK [testnode : Create the teuthology user.] ********************************** 2020-03-19T21:13:05.255 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:13:05.294 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:13:05 +0000 (0:00:00.076) 0:01:53.898 ******** 2020-03-19T21:13:05.320 INFO:teuthology.task.ansible.out: TASK [testnode : Delete the teuthology users password.] ************************ 2020-03-19T21:13:05.321 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:13:05.375 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:13:05 +0000 (0:00:00.080) 0:01:53.978 ******** 2020-03-19T21:13:05.598 INFO:teuthology.task.ansible.out: TASK [testnode : Add a user for xfstests to test user quotas.] ***************** 2020-03-19T21:13:05.599 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 10101, "home": "/home/fsgqa", "move_home": false, "name": "fsgqa", "shell": "/bin/bash", "state": "present", "uid": 10101} 2020-03-19T21:13:05.633 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:13:05 +0000 (0:00:00.258) 0:01:54.236 ******** 2020-03-19T21:13:06.327 INFO:teuthology.task.ansible.out: TASK [testnode : Set a high max open files limit for the teuthology user.] ***** 2020-03-19T21:13:06.328 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => {"changed": false, "checksum": "dc140f4bb26594b8d5e0753bab8d93c0cced7772", "dest": "/etc/security/limits.d/ubuntu.conf", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/security/limits.d/ubuntu.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 112, "state": "file", "uid": 0} 2020-03-19T21:13:06.369 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:13:06 +0000 (0:00:00.736) 0:01:54.973 ******** 2020-03-19T21:13:06.407 INFO:teuthology.task.ansible.out: TASK [testnode : Set hostname var] ********************************************* 2020-03-19T21:13:06.408 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => {"ansible_facts": {"hostname": "smithi170"}, "changed": false} 2020-03-19T21:13:06.449 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:13:06 +0000 (0:00:00.079) 0:01:55.053 ******** 2020-03-19T21:13:07.517 INFO:teuthology.task.ansible.out: TASK [testnode : Set the system's hostname] ************************************ 2020-03-19T21:13:07.519 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => {"ansible_facts": {"ansible_domain": "", "ansible_fqdn": "smithi170", "ansible_hostname": "smithi170", "ansible_nodename": "smithi170"}, "changed": false, "name": "smithi170"} 2020-03-19T21:13:07.559 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:13:07 +0000 (0:00:01.109) 0:01:56.163 ******** 2020-03-19T21:13:08.038 INFO:teuthology.task.ansible.out: TASK [testnode : Upload distro major version specific sshd_config] ************* 2020-03-19T21:13:08.039 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => {"changed": false, "checksum": "a5e550b2aa74f6355e926d70d73a50ad9ab4b4be", "dest": "/etc/ssh/sshd_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/sshd_config", "secontext": "system_u:object_r:etc_t:s0", "size": 1043, "state": "file", "uid": 0} 2020-03-19T21:13:08.076 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:13:08 +0000 (0:00:00.517) 0:01:56.680 ******** 2020-03-19T21:13:08.561 INFO:teuthology.task.ansible.out: TASK [testnode : Upload ssh_config] ******************************************** 2020-03-19T21:13:08.561 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => {"changed": false, "checksum": "58d78b02605c8368f1913de12e145f4ad930fafc", "dest": "/etc/ssh/ssh_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/ssh_config", "secontext": "system_u:object_r:etc_t:s0", "size": 570, "state": "file", "uid": 0} 2020-03-19T21:13:08.607 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:13:08 +0000 (0:00:00.530) 0:01:57.210 ******** 2020-03-19T21:13:09.243 INFO:teuthology.task.ansible.out: TASK [testnode : Add ssh pubkeys] ********************************************** 2020-03-19T21:13:09.244 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => {"attempts": 1, "changed": false, "comment": null, "exclusive": false, "follow": false, "key": "https://raw.githubusercontent.com/ceph/keys/autogenerated/ssh/@all.pub", "key_options": null, "keyfile": "/home/ubuntu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ubuntu", "validate_certs": true} 2020-03-19T21:13:09.280 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:13:09 +0000 (0:00:00.673) 0:01:57.884 ******** 2020-03-19T21:13:09.751 INFO:teuthology.task.ansible.out: TASK [testnode : Create remote.conf] ******************************************* 2020-03-19T21:13:09.752 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => {"changed": false, "checksum": "68d0b8ee2f88a77895f257aa69f32a42e47f455b", "dest": "/etc/security/limits.d/remote.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/security/limits.d/remote.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 116, "state": "file", "uid": 0} 2020-03-19T21:13:09.794 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:13:09 +0000 (0:00:00.513) 0:01:58.397 ******** 2020-03-19T21:13:09.997 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2020-03-19T21:13:09.998 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => {"changed": false, "dest": "/etc/fuse.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "secontext": "unconfined_u:object_r:etc_t:s0", "size": 0, "state": "file", "uid": 0} 2020-03-19T21:13:10.052 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:13:10 +0000 (0:00:00.258) 0:01:58.656 ******** 2020-03-19T21:13:10.253 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* 2020-03-19T21:13:10.254 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => {"changed": false, "gid": 36, "name": "kvm", "state": "present", "system": false} 2020-03-19T21:13:10.316 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:13:10 +0000 (0:00:00.263) 0:01:58.919 ******** 2020-03-19T21:13:10.549 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2020-03-19T21:13:10.550 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => {"append": true, "changed": false, "comment": "", "group": 1211, "groups": "kvm,disk", "home": "/home/ubuntu", "move_home": false, "name": "ubuntu", "shell": "/bin/bash", "state": "present", "uid": 1000} 2020-03-19T21:13:10.586 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:13:10 +0000 (0:00:00.269) 0:01:59.189 ******** 2020-03-19T21:13:11.065 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** 2020-03-19T21:13:11.066 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => {"changed": false, "checksum": "be3223c8b0917bde02657a5f9ddd95277cc3f812", "dest": "/etc/sudoers", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers", "secontext": "system_u:object_r:etc_t:s0", "size": 1808, "state": "file", "uid": 0} 2020-03-19T21:13:11.114 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:13:11 +0000 (0:00:00.528) 0:01:59.718 ******** 2020-03-19T21:13:11.613 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** 2020-03-19T21:13:11.614 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => {"changed": false, "checksum": "12a2e17405b9e00ea2061e25929905d08e5fe19f", "dest": "/etc/security/limits.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/security/limits.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 854, "state": "file", "uid": 0} 2020-03-19T21:13:11.669 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:13:11 +0000 (0:00:00.554) 0:02:00.272 ******** 2020-03-19T21:13:13.568 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2020-03-19T21:13:13.569 INFO:teuthology.task.ansible.out: [WARNING]: Consider using the yum, dnf or zypper module rather than running 'rpm'. If you need to use command because yum, dnf or zypper is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message. 2020-03-19T21:13:13.570 INFO:teuthology.task.ansible.out:changed: [smithi170.front.sepia.ceph.com] => {"changed": true, "cmd": ["rpm", "--rebuilddb"], "delta": "0:00:01.704457", "end": "2020-03-19 21:13:13.546329", "rc": 0, "start": "2020-03-19 21:13:11.841872", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-03-19T21:13:13.609 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:13:13 +0000 (0:00:01.940) 0:02:02.212 ******** 2020-03-19T21:13:13.655 INFO:teuthology.task.ansible.out: TASK [testnode : Configure local mirrorlists] ********************************** 2020-03-19T21:13:13.656 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:13:13.690 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:13:13 +0000 (0:00:00.080) 0:02:02.293 ******** 2020-03-19T21:13:14.218 INFO:teuthology.task.ansible.out: TASK [testnode : Configure common additional repos in /etc/yum.repos.d/] ******* 2020-03-19T21:13:14.219 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={'value': {u'gpgcheck': 0, u'enabled': 1, u'name': u'lab-extras', u'baseurl': u'http://apt-mirror.front.sepia.ceph.com/lab-extras/8/'}, 'key': u'lab-extras'}) => {"ansible_loop_var": "item", "changed": false, "checksum": "a0e2ac0628a264e14b96b1c57627a3aa344a6c52", "dest": "/etc/yum.repos.d/lab-extras.repo", "gid": 0, "group": "root", "item": {"key": "lab-extras", "value": {"baseurl": "http://apt-mirror.front.sepia.ceph.com/lab-extras/8/", "enabled": 1, "gpgcheck": 0, "name": "lab-extras"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/lab-extras.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 203, "state": "file", "uid": 0} 2020-03-19T21:13:14.262 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:13:14 +0000 (0:00:00.571) 0:02:02.865 ******** 2020-03-19T21:13:14.287 INFO:teuthology.task.ansible.out: TASK [testnode : Configure version specific repos in /etc/yum.repos.d/] ******** 2020-03-19T21:13:14.325 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:13:14 +0000 (0:00:00.063) 0:02:02.928 ******** 2020-03-19T21:13:15.307 INFO:teuthology.task.ansible.out: TASK [testnode : Enable copr repos] ******************************************** 2020-03-19T21:13:15.307 INFO:teuthology.task.ansible.out:changed: [smithi170.front.sepia.ceph.com] => (item=ktdreyer/ceph-el8) => {"ansible_loop_var": "item", "changed": true, "cmd": ["dnf", "-y", "copr", "enable", "ktdreyer/ceph-el8"], "delta": "0:00:00.760489", "end": "2020-03-19 21:13:15.275094", "item": "ktdreyer/ceph-el8", "rc": 0, "start": "2020-03-19 21:13:14.514605", "stderr": "", "stderr_lines": [], "stdout": "Updating Subscription Management repositories.\nRepository successfully enabled.", "stdout_lines": ["Updating Subscription Management repositories.", "Repository successfully enabled."], "warnings": ["Consider using the dnf module rather than running 'dnf'. If you need to use command because dnf is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message."]} 2020-03-19T21:13:15.309 INFO:teuthology.task.ansible.out: [WARNING]: Consider using the dnf module rather than running 'dnf'. If you need to use command because dnf is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message. 2020-03-19T21:13:15.345 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:13:15 +0000 (0:00:01.020) 0:02:03.949 ******** 2020-03-19T21:13:15.371 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG] ************************************************** 2020-03-19T21:13:15.372 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:13:15.408 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:13:15 +0000 (0:00:00.062) 0:02:04.011 ******** 2020-03-19T21:13:15.443 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG keys] ********************************************* 2020-03-19T21:13:15.445 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} 2020-03-19T21:13:15.455 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} 2020-03-19T21:13:15.492 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:13:15 +0000 (0:00:00.084) 0:02:04.096 ******** 2020-03-19T21:13:15.517 INFO:teuthology.task.ansible.out: TASK [testnode : Clean yum cache] ********************************************** 2020-03-19T21:13:15.519 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:13:15.554 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:13:15 +0000 (0:00:00.061) 0:02:04.158 ******** 2020-03-19T21:13:15.579 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-03-19T21:13:15.581 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:13:15.617 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:13:15 +0000 (0:00:00.062) 0:02:04.221 ******** 2020-03-19T21:13:15.652 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure yum_utils is present.] ********************************* 2020-03-19T21:13:15.654 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:13:15.688 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:13:15 +0000 (0:00:00.070) 0:02:04.291 ******** 2020-03-19T21:13:15.723 INFO:teuthology.task.ansible.out: TASK [testnode : Removing saved yum transactions] ****************************** skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:13:15.760 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:13:15 +0000 (0:00:00.071) 0:02:04.363 ******** 2020-03-19T21:13:15.974 INFO:teuthology.task.ansible.out: TASK [testnode : Check if ceph-debuginfo is installed] ************************* 2020-03-19T21:13:15.975 INFO:teuthology.task.ansible.out:fatal: [smithi170.front.sepia.ceph.com]: FAILED! => {"changed": false, "cmd": ["rpm", "-q", "ceph-debuginfo"], "delta": "0:00:00.006765", "end": "2020-03-19 21:13:15.949690", "msg": "non-zero return code", "rc": 1, "start": "2020-03-19 21:13:15.942925", "stderr": "", "stderr_lines": [], "stdout": "package ceph-debuginfo is not installed", "stdout_lines": ["package ceph-debuginfo is not installed"]} ...ignoring 2020-03-19T21:13:16.011 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:13:16 +0000 (0:00:00.251) 0:02:04.614 ******** 2020-03-19T21:13:16.046 INFO:teuthology.task.ansible.out: TASK [testnode : Work around https://bugzilla.redhat.com/show_bug.cgi?id=1234967] *** 2020-03-19T21:13:16.048 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:13:16.082 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:13:16 +0000 (0:00:00.071) 0:02:04.686 ******** 2020-03-19T21:13:22.858 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ 2020-03-19T21:13:22.860 INFO:teuthology.task.ansible.out:changed: [smithi170.front.sepia.ceph.com] => {"changed": true, "msg": "", "rc": 0, "results": ["Removed: librados2-1:12.2.7-9.el8.x86_64", "Removed: librbd1-1:12.2.7-9.el8.x86_64", "Removed: qemu-kvm-block-rbd-15:2.12.0-88.module+el8.1.0+5149+3ff2765e.2.x86_64"]} 2020-03-19T21:13:22.907 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:13:22 +0000 (0:00:06.825) 0:02:11.511 ******** 2020-03-19T21:13:29.957 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* 2020-03-19T21:13:29.957 INFO:teuthology.task.ansible.out:changed: [smithi170.front.sepia.ceph.com] => {"changed": true, "msg": "", "rc": 0, "results": ["Removed: tuned-2.12.0-3.el8_1.1.noarch", "Removed: hdparm-9.54-2.el8.x86_64", "Removed: boost-program-options-1.66.0-6.el8.x86_64", "Removed: boost-random-1.66.0-6.el8.x86_64"]} 2020-03-19T21:13:29.997 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:13:29 +0000 (0:00:07.089) 0:02:18.600 ******** 2020-03-19T21:13:41.588 INFO:teuthology.task.ansible.out: TASK [testnode : Base group workaround for BZ1782899] ************************** 2020-03-19T21:13:41.588 INFO:teuthology.task.ansible.out:changed: [smithi170.front.sepia.ceph.com] => {"changed": true, "cmd": "dnf -y groupinstall base --nobest && dnf -y group upgrade base", "delta": "0:00:11.380673", "end": "2020-03-19 21:13:41.561047", "rc": 0, "start": "2020-03-19 21:13:30.180374", "stderr": "", "stderr_lines": [], "stdout": "Updating Subscription Management repositories.\nCopr repo for ceph-el8 owned by ktdreyer 71 kB/s | 3.3 kB 00:00 \nExtra Packages for Enterprise Linux 92 kB/s | 5.3 kB 00:00 \nlab-extras 1.2 MB/s | 2.9 kB 00:00 \nRed Hat Enterprise Linux 8 for x86_64 - BaseOS 58 kB/s | 2.4 kB 00:00 \nRed Hat Enterprise Linux 8 for x86_64 - AppStre 90 kB/s | 2.8 kB 00:00 \nDependencies resolved.\n================================================================================\n Package Architecture Version Repository Size\n================================================================================\nInstalling Groups:\n Base \n\nTransaction Summary\n================================================================================\n\nComplete!\nUpdating Subscription Management repositories.\nRed Hat Enterprise Linux 8 for x86_64 - BaseOS 63 kB/s | 2.4 kB 00:00 \nRed Hat Enterprise Linux 8 for x86_64 - AppStre 74 kB/s | 2.8 kB 00:00 \nDependencies resolved.\n================================================================================\n Package Architecture Version Repository Size\n================================================================================\nUpgrading Groups:\n Base \n\nTransaction Summary\n================================================================================\n\nComplete!", "stdout_lines": ["Updating Subscription Management repositories.", " 2020-03-19T21:13:41.590 INFO:teuthology.task.ansible.out:Copr repo for ceph-el8 owned by ktdreyer 71 kB/s | 3.3 kB 00:00 ", "Extra Packages for Enterprise Linux 92 kB/s | 5.3 kB 00:00 ", "lab-extras 1.2 MB/s | 2.9 kB 00:00 ", "Red Hat Enterprise Linux 8 for x86_64 - BaseOS 58 kB/s | 2.4 kB 00:00 ", "Red Hat Enterprise Linux 8 for x86_64 - AppStre 90 kB/s | 2.8 kB 00:00 ", "Dependencies resolved.", "================================================================================", " Package Architecture Version Repository Size", "================================================================================", "Installing Groups:", " Base ", "", "Transaction Summary", "================================================================================", "", "Complete!", "Updating Subscription Management repositories.", "Red Hat Enterprise Linux 8 for x86_64 - BaseOS 63 kB/s | 2.4 kB 00:00 ", "Red Hat Enterprise Linux 8 for x86_64 - AppStre 74 kB/s | 2.8 kB 00:00 ", "Dependencies resolved.", "================================================================================", " Package Architecture Version Repository Size", "================================================================================", "Upgrading Groups:", " Base ", "", "Transaction Summary", "================================================================================", "", "Complete!"]} 2020-03-19T21:13:41.625 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:13:41 +0000 (0:00:11.628) 0:02:30.229 ******** 2020-03-19T21:13:54.961 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2020-03-19T21:13:54.962 INFO:teuthology.task.ansible.out:changed: [smithi170.front.sepia.ceph.com] => {"changed": true, "msg": "", "rc": 0, "results": ["Group core installed.", "Group base installed.", "Installed: dnf-utils", "Installed: git-all", "Installed: sysstat", "Installed: libedit", "Installed: boost-thread", "Installed: xfsprogs", "Installed: gdisk", "Installed: parted", "Installed: libgcrypt", "Installed: fuse-libs", "Installed: openssl", "Installed: libuuid", "Installed: attr", "Installed: ant", "Installed: lsof", "Installed: gettext", "Installed: bc", "Installed: xfsdump", "Installed: blktrace", "Installed: usbredir", "Installed: podman", "Installed: podman-docker", "Installed: libev-devel", "Installed: valgrind", "Installed: nfs-utils", "Installed: ncurses-devel", "Installed: gcc", "Installed: git", "Installed: python3-nose", "Installed: python3-virtualenv", "Installed: genisoimage", "Installed: qemu-img", "Installed: qemu-kvm-core", "Installed: qemu-kvm-block-rbd", "Installed: libacl-devel", "Installed: autoconf", "Installed: gdb", "Installed: librados2-1:12.2.7-9.el8.x86_64", "Installed: boost-random-1.66.0-6.el8.x86_64", "Installed: librbd1-1:12.2.7-9.el8.x86_64", "Installed: boost-program-options-1.66.0-6.el8.x86_64", "Installed: tuned-2.12.0-3.el8_1.1.noarch", "Installed: qemu-kvm-block-rbd-15:2.12.0-88.module+el8.1.0+5149+3ff2765e.2.x86_64", "Installed: hdparm-9.54-2.el8.x86_64"]} 2020-03-19T21:13:55.015 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:13:55 +0000 (0:00:13.389) 0:02:43.618 ******** 2020-03-19T21:13:57.124 INFO:teuthology.task.ansible.out: TASK [testnode : Install epel packages] **************************************** 2020-03-19T21:13:57.125 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": ["Installed: dbench"]} 2020-03-19T21:13:57.161 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:13:57 +0000 (0:00:02.146) 0:02:45.765 ******** 2020-03-19T21:13:59.152 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2020-03-19T21:13:59.153 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": []} 2020-03-19T21:13:59.199 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:13:59 +0000 (0:00:02.037) 0:02:47.802 ******** 2020-03-19T21:13:59.234 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2020-03-19T21:13:59.236 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:13:59.275 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:13:59 +0000 (0:00:00.076) 0:02:47.879 ******** 2020-03-19T21:13:59.613 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable firewalld] *********************************** 2020-03-19T21:13:59.617 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => {"changed": false, "enabled": false, "name": "firewalld", "state": "stopped", "status": {"ActiveEnterTimestampMonotonic": "0", "ActiveExitTimestampMonotonic": "0", "ActiveState": "inactive", "After": "basic.target dbus.service system.slice polkit.service sysinit.target dbus.socket", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "no", "AssertTimestampMonotonic": "0", "Before": "shutdown.target network-pre.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "BusName": "org.fedoraproject.FirewallD1", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "no", "ConditionTimestampMonotonic": "0", "ConfigurationDirectoryMode": "0755", "Conflicts": "ebtables.service shutdown.target ipset.service ip6tables.service iptables.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "firewalld - dynamic firewall daemon", "DevicePolicy": "auto", "Documentation": "man:firewalld(1)", "DynamicUser": "no", "EnvironmentFiles": "/etc/sysconfig/firewalld (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTim 2020-03-19T21:13:59.618 INFO:teuthology.task.ansible.out:estampMonotonic": "0", "ExecMainPID": "0", "ExecMainStartTimestampMonotonic": "0", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/firewalld ; argv[]=/usr/sbin/firewalld --nofork --nopid $FIREWALLD_ARGS ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/firewalld.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "firewalld.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestampMonotonic": "0", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "mixed", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "infinity", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "16777216", "LimitMEMLOCKSoft": "16777216", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "127442", "LimitNPROCSoft": "127442", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "127442", 2020-03-19T21:13:59.620 INFO:teuthology.task.ansible.out:"LimitSIGPENDINGSoft": "127442", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "0", "MemoryAccounting": "yes", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "firewalld.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateMounts": "no", "PrivateNetwork": "no", "PrivateTmp": "no", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "sysinit.target dbus.socket system.slice", "Restart": "no", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "null", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "null", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestampMonotonic": "0", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "dead", "SuccessAction": "none", "SyslogFacility": "3 2020-03-19T21:13:59.621 INFO:teuthology.task.ansible.out:", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "[not set]", "TasksMax": "26213", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "dbus", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "disabled", "UtmpMode": "init", "Wants": "network-pre.target", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} 2020-03-19T21:13:59.661 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:13:59 +0000 (0:00:00.385) 0:02:48.265 ******** 2020-03-19T21:13:59.967 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable iptables] ************************************ 2020-03-19T21:13:59.968 INFO:teuthology.task.ansible.out:fatal: [smithi170.front.sepia.ceph.com]: FAILED! => {"changed": false, "msg": "Could not find the requested service iptables: host"} ...ignoring 2020-03-19T21:14:00.004 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:13:59 +0000 (0:00:00.343) 0:02:48.608 ******** 2020-03-19T21:14:00.728 INFO:teuthology.task.ansible.out: TASK [testnode : Enable SELinux] *********************************************** 2020-03-19T21:14:00.729 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => {"changed": false, "configfile": "/etc/selinux/config", "msg": "", "policy": "targeted", "reboot_required": false, "state": "permissive"} 2020-03-19T21:14:00.798 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:00 +0000 (0:00:00.793) 0:02:49.402 ******** 2020-03-19T21:14:00.824 INFO:teuthology.task.ansible.out: TASK [testnode : Check for custom repos] *************************************** 2020-03-19T21:14:00.825 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:14:00.861 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:00 +0000 (0:00:00.062) 0:02:49.464 ******** 2020-03-19T21:14:00.887 INFO:teuthology.task.ansible.out: TASK [testnode : Remove custom repos] ****************************************** 2020-03-19T21:14:00.945 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:00 +0000 (0:00:00.083) 0:02:49.548 ******** 2020-03-19T21:14:00.973 INFO:teuthology.task.ansible.out: TASK [testnode : Set apt preferences] ****************************************** 2020-03-19T21:14:00.975 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:14:01.009 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:01 +0000 (0:00:00.064) 0:02:49.613 ******** 2020-03-19T21:14:01.035 INFO:teuthology.task.ansible.out: TASK [testnode : Add sources list] ********************************************* 2020-03-19T21:14:01.036 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:14:01.074 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:01 +0000 (0:00:00.065) 0:02:49.678 ******** 2020-03-19T21:14:01.112 INFO:teuthology.task.ansible.out: TASK [testnode : Install apt keys] ********************************************* 2020-03-19T21:14:01.113 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} 2020-03-19T21:14:01.123 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} 2020-03-19T21:14:01.160 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:01 +0000 (0:00:00.085) 0:02:49.764 ******** 2020-03-19T21:14:01.197 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-apt] ******************************************* 2020-03-19T21:14:01.198 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:14:01.235 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:01 +0000 (0:00:00.074) 0:02:49.838 ******** 2020-03-19T21:14:01.261 INFO:teuthology.task.ansible.out: TASK [testnode : Add local apt repos.] ***************************************** 2020-03-19T21:14:01.304 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:01 +0000 (0:00:00.068) 0:02:49.907 ******** 2020-03-19T21:14:01.329 INFO:teuthology.task.ansible.out: TASK [testnode : Blast the apt cache.] ***************************************** 2020-03-19T21:14:01.331 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:14:01.369 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:01 +0000 (0:00:00.065) 0:02:49.973 ******** 2020-03-19T21:14:01.405 INFO:teuthology.task.ansible.out: TASK [testnode : Update apt cache.] ******************************************** 2020-03-19T21:14:01.407 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:14:01.465 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:01 +0000 (0:00:00.095) 0:02:50.068 ******** 2020-03-19T21:14:01.492 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure packages are not present.] ***************************** 2020-03-19T21:14:01.493 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:14:01.548 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:01 +0000 (0:00:00.083) 0:02:50.151 ******** 2020-03-19T21:14:01.575 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2020-03-19T21:14:01.577 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:14:01.629 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:01 +0000 (0:00:00.080) 0:02:50.232 ******** 2020-03-19T21:14:01.655 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade non aarch64 packages] ********************************* 2020-03-19T21:14:01.656 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:14:01.694 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:01 +0000 (0:00:00.065) 0:02:50.298 ******** 2020-03-19T21:14:01.719 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2020-03-19T21:14:01.721 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:14:01.757 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:01 +0000 (0:00:00.062) 0:02:50.360 ******** 2020-03-19T21:14:01.782 INFO:teuthology.task.ansible.out: TASK [testnode : Install non aarch64 packages] ********************************* 2020-03-19T21:14:01.783 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:14:01.819 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:01 +0000 (0:00:00.061) 0:02:50.422 ******** 2020-03-19T21:14:01.845 INFO:teuthology.task.ansible.out: TASK [testnode : Use perl to add user_xattr to the root mount options in fstab.] *** 2020-03-19T21:14:01.847 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:14:01.895 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:01 +0000 (0:00:00.076) 0:02:50.499 ******** 2020-03-19T21:14:01.921 INFO:teuthology.task.ansible.out: TASK [testnode : Enable xattr for this boot.] ********************************** 2020-03-19T21:14:01.922 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:14:01.957 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:01 +0000 (0:00:00.062) 0:02:50.561 ******** 2020-03-19T21:14:01.983 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure fuse group exists.] ************************************ 2020-03-19T21:14:01.984 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:14:02.024 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:02 +0000 (0:00:00.066) 0:02:50.628 ******** 2020-03-19T21:14:02.053 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/fuse.conf.] *************************************** 2020-03-19T21:14:02.054 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:14:02.105 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:02 +0000 (0:00:00.080) 0:02:50.709 ******** 2020-03-19T21:14:02.194 INFO:teuthology.task.ansible.out: TASK [testnode : Add teuthology user to groups fuse, kvm and disk.] ************ skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Thursday 19 March 2020 21:14:02 +0000 (0:00:00.065) 0:02:50.774 ******** 2020-03-19T21:14:02.212 INFO:teuthology.task.ansible.out: TASK [testnode : Set up static IP in /etc/hosts.] ****************************** 2020-03-19T21:14:02.214 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:14:02.259 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:02 +0000 (0:00:00.088) 0:02:50.863 ******** 2020-03-19T21:14:02.285 INFO:teuthology.task.ansible.out: TASK [testnode : Stop apache2] ************************************************* 2020-03-19T21:14:02.287 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:14:02.328 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:02 +0000 (0:00:00.068) 0:02:50.931 ******** 2020-03-19T21:14:02.354 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2020-03-19T21:14:02.355 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:14:02.395 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:02 +0000 (0:00:00.066) 0:02:50.998 ******** 2020-03-19T21:14:02.421 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* 2020-03-19T21:14:02.423 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:14:02.476 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:02 +0000 (0:00:00.080) 0:02:51.079 ******** 2020-03-19T21:14:02.502 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2020-03-19T21:14:02.503 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:14:02.547 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:02 +0000 (0:00:00.071) 0:02:51.151 ******** 2020-03-19T21:14:02.580 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** 2020-03-19T21:14:02.582 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:14:02.626 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:02 +0000 (0:00:00.078) 0:02:51.229 ******** 2020-03-19T21:14:02.655 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:14:02.706 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:02 +0000 (0:00:00.075) 0:02:51.305 ******** 2020-03-19T21:14:02.730 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2020-03-19T21:14:02.732 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:14:02.790 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:02 +0000 (0:00:00.088) 0:02:51.394 ******** 2020-03-19T21:14:02.818 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ 2020-03-19T21:14:02.820 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:14:02.864 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:02 +0000 (0:00:00.073) 0:02:51.467 ******** 2020-03-19T21:14:02.890 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* 2020-03-19T21:14:02.892 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:14:02.932 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:02 +0000 (0:00:00.068) 0:02:51.535 ******** 2020-03-19T21:14:02.958 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-03-19T21:14:02.958 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:14:02.994 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:02 +0000 (0:00:00.062) 0:02:51.598 ******** 2020-03-19T21:14:03.020 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2020-03-19T21:14:03.022 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:14:03.059 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:03 +0000 (0:00:00.064) 0:02:51.662 ******** 2020-03-19T21:14:03.106 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2020-03-19T21:14:03.107 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:14:03.149 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:03 +0000 (0:00:00.089) 0:02:51.752 ******** 2020-03-19T21:14:03.185 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2020-03-19T21:14:03.187 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:14:03.223 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:03 +0000 (0:00:00.074) 0:02:51.826 ******** 2020-03-19T21:14:03.250 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2020-03-19T21:14:03.251 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:14:03.289 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:03 +0000 (0:00:00.066) 0:02:51.893 ******** 2020-03-19T21:14:03.315 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2020-03-19T21:14:03.317 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:14:03.357 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:03 +0000 (0:00:00.068) 0:02:51.961 ******** 2020-03-19T21:14:03.383 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2020-03-19T21:14:03.384 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:14:03.421 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:03 +0000 (0:00:00.063) 0:02:52.025 ******** 2020-03-19T21:14:03.447 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2020-03-19T21:14:03.449 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:14:03.485 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:03 +0000 (0:00:00.063) 0:02:52.088 ******** 2020-03-19T21:14:03.511 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2020-03-19T21:14:03.512 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:14:03.549 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:03 +0000 (0:00:00.063) 0:02:52.152 ******** 2020-03-19T21:14:03.575 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2020-03-19T21:14:03.576 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:14:03.614 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:03 +0000 (0:00:00.064) 0:02:52.217 ******** 2020-03-19T21:14:03.639 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2020-03-19T21:14:03.640 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:14:03.681 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:03 +0000 (0:00:00.067) 0:02:52.284 ******** 2020-03-19T21:14:03.707 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2020-03-19T21:14:03.709 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:14:03.752 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:03 +0000 (0:00:00.070) 0:02:52.355 ******** 2020-03-19T21:14:03.778 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2020-03-19T21:14:03.778 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:14:03.818 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:03 +0000 (0:00:00.066) 0:02:52.422 ******** 2020-03-19T21:14:03.844 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2020-03-19T21:14:03.846 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:14:03.893 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:03 +0000 (0:00:00.075) 0:02:52.497 ******** 2020-03-19T21:14:03.919 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2020-03-19T21:14:03.920 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:14:03.970 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:03 +0000 (0:00:00.076) 0:02:52.573 ******** 2020-03-19T21:14:03.999 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2020-03-19T21:14:04.001 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:14:04.047 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:04 +0000 (0:00:00.077) 0:02:52.651 ******** 2020-03-19T21:14:04.074 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2020-03-19T21:14:04.075 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:14:04.112 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:04 +0000 (0:00:00.064) 0:02:52.716 ******** 2020-03-19T21:14:04.139 INFO:teuthology.task.ansible.out: TASK [testnode : Set grub config.] ********************************************* 2020-03-19T21:14:04.141 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:14:04.184 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:04 +0000 (0:00:00.071) 0:02:52.787 ******** 2020-03-19T21:14:04.211 INFO:teuthology.task.ansible.out: TASK [testnode : Remove /etc/ceph] ********************************************* 2020-03-19T21:14:04.212 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:14:04.248 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:04 +0000 (0:00:00.064) 0:02:52.852 ******** 2020-03-19T21:14:04.279 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2020-03-19T21:14:04.280 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:14:04.331 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:04 +0000 (0:00:00.082) 0:02:52.934 ******** 2020-03-19T21:14:04.357 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2020-03-19T21:14:04.361 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:14:04.402 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:04 +0000 (0:00:00.071) 0:02:53.006 ******** 2020-03-19T21:14:04.428 INFO:teuthology.task.ansible.out: TASK [testnode : Upload weekly kernel-clean crontab.] ************************** 2020-03-19T21:14:04.430 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:14:04.470 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:04 +0000 (0:00:00.067) 0:02:53.073 ******** 2020-03-19T21:14:04.495 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/grub.d/02_force_timeout.] ************************* 2020-03-19T21:14:04.497 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:14:04.537 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:04 +0000 (0:00:00.067) 0:02:53.141 ******** 2020-03-19T21:14:04.565 INFO:teuthology.task.ansible.out: TASK [testnode : Enable kernel modules to load at boot time.] ****************** 2020-03-19T21:14:04.566 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:14:04.604 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:04 +0000 (0:00:00.066) 0:02:53.207 ******** 2020-03-19T21:14:04.629 INFO:teuthology.task.ansible.out: TASK [testnode : Enabling auto-fsck fix to prevent boot hangup.] *************** 2020-03-19T21:14:04.630 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:14:04.667 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:04 +0000 (0:00:00.063) 0:02:53.271 ******** 2020-03-19T21:14:04.693 INFO:teuthology.task.ansible.out: TASK [testnode : Work around broken wget on wheezy.] *************************** 2020-03-19T21:14:04.695 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:14:04.736 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:04 +0000 (0:00:00.068) 0:02:53.339 ******** 2020-03-19T21:14:04.738 INFO:teuthology.task.ansible.out: 2020-03-19T21:14:04.763 INFO:teuthology.task.ansible.out: TASK [testnode : Stop collectl] ************************************************ 2020-03-19T21:14:04.764 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:14:04.803 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:04 +0000 (0:00:00.066) 0:02:53.406 ******** 2020-03-19T21:14:04.829 INFO:teuthology.task.ansible.out: TASK [testnode : Add PATH to the teuthology_user .bashrc.] ********************* 2020-03-19T21:14:04.831 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:14:04.868 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:04 +0000 (0:00:00.065) 0:02:53.472 ******** 2020-03-19T21:14:04.894 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if we need to edit /etc/profile.] **************** 2020-03-19T21:14:04.896 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:14:04.938 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:04 +0000 (0:00:00.070) 0:02:53.542 ******** 2020-03-19T21:14:04.964 INFO:teuthology.task.ansible.out: TASK [testnode : Update /etc/profile if needed.] ******************************* 2020-03-19T21:14:04.966 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:14:05.000 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:04 +0000 (0:00:00.061) 0:02:53.603 ******** 2020-03-19T21:14:05.027 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-03-19T21:14:05.028 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => {"ansible_facts": {"nvme_card_present": false}, "changed": false} 2020-03-19T21:14:05.069 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:05 +0000 (0:00:00.069) 0:02:53.673 ******** 2020-03-19T21:14:05.127 INFO:teuthology.task.ansible.out: TASK [testnode : Check for NVMe drive] ***************************************** 2020-03-19T21:14:05.129 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item=sda) => {"ansible_loop_var": "item", "changed": false, "item": "sda", "skip_reason": "Conditional result was False"} 2020-03-19T21:14:05.152 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item=nvme0n1) => {"ansible_facts": {"nvme_card_present": true}, "ansible_loop_var": "item", "changed": false, "item": "nvme0n1"} 2020-03-19T21:14:05.163 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item=dm-4) => {"ansible_loop_var": "item", "changed": false, "item": "dm-4", "skip_reason": "Conditional result was False"} 2020-03-19T21:14:05.176 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item=dm-2) => {"ansible_loop_var": "item", "changed": false, "item": "dm-2", "skip_reason": "Conditional result was False"} 2020-03-19T21:14:05.185 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item=dm-3) => {"ansible_loop_var": "item", "changed": false, "item": "dm-3", "skip_reason": "Conditional result was False"} 2020-03-19T21:14:05.186 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item=dm-0) => {"ansible_loop_var": "item", "changed": false, "item": "dm-0", "skip_reason": "Conditional result was False"} 2020-03-19T21:14:05.196 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item=dm-1) => {"ansible_loop_var": "item", "changed": false, "item": "dm-1", "skip_reason": "Conditional result was False"} 2020-03-19T21:14:05.241 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:05 +0000 (0:00:00.171) 0:02:53.845 ******** 2020-03-19T21:14:05.402 INFO:teuthology.task.ansible.out: TASK [testnode : Check for teuthology-lock command] **************************** 2020-03-19T21:14:05.403 INFO:teuthology.task.ansible.out:changed: [smithi170.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "which teuthology-lock", "delta": "0:00:00.003203", "end": "2020-03-19 21:14:05.365387", "rc": 0, "start": "2020-03-19 21:14:05.362184", "stderr": "", "stderr_lines": [], "stdout": "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock", "stdout_lines": ["/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock"]} 2020-03-19T21:14:05.446 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:05 +0000 (0:00:00.204) 0:02:54.050 ******** 2020-03-19T21:14:05.493 INFO:teuthology.task.ansible.out: TASK [testnode : Mark system down if NVMe card missing] ************************ 2020-03-19T21:14:05.494 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:14:05.537 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:05 +0000 (0:00:00.090) 0:02:54.140 ******** 2020-03-19T21:14:05.565 INFO:teuthology.task.ansible.out: TASK [testnode : Update description in paddles lock DB if NVMe card missing] *** 2020-03-19T21:14:05.566 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:14:05.612 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:05 +0000 (0:00:00.075) 0:02:54.215 ******** 2020-03-19T21:14:05.648 INFO:teuthology.task.ansible.out: TASK [testnode : Fail rest of playbook due to missing NVMe card] *************** 2020-03-19T21:14:05.649 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:14:05.687 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:05 +0000 (0:00:00.075) 0:02:54.290 ******** 2020-03-19T21:14:05.713 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure apt dependencies are installed] ********************* 2020-03-19T21:14:05.715 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:14:05.759 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:05 +0000 (0:00:00.071) 0:02:54.362 ******** 2020-03-19T21:14:07.774 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure rpm dependencies are installed] ********************* ok: [smithi170.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": ["Installed: gdisk", "Installed: device-mapper"]} 2020-03-19T21:14:07.809 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:07 +0000 (0:00:02.050) 0:02:56.413 ******** 2020-03-19T21:14:07.845 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2020-03-19T21:14:07.845 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item={u'block_used': 12916307, u'uuid': u'51c8dc47-a34c-4398-9853-098c2389802a', u'size_total': 983429562368, u'block_total': 240095108, u'mount': u'/', u'block_available': 227178801, u'size_available': 930524368896, u'fstype': u'ext4', u'inode_total': 61054976, u'inode_available': 60987191, u'device': u'/dev/sda1', u'inode_used': 67785, u'block_size': 4096, u'options': u'rw,seclabel,relatime'}) => {"ansible_facts": {"root_disk": "/dev/sda"}, "ansible_loop_var": "item", "changed": false, "item": {"block_available": 227178801, "block_size": 4096, "block_total": 240095108, "block_used": 12916307, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60987191, "inode_total": 61054976, "inode_used": 67785, "mount": "/", "options": "rw,seclabel,relatime", "size_available": 930524368896, "size_total": 983429562368, "uuid": "51c8dc47-a34c-4398-9853-098c2389802a"}} 2020-03-19T21:14:07.882 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:07 +0000 (0:00:00.072) 0:02:56.485 ******** 2020-03-19T21:14:08.094 INFO:teuthology.task.ansible.out: TASK [testnode : Compile list of non-root partitions] ************************** 2020-03-19T21:14:08.096 INFO:teuthology.task.ansible.out:changed: [smithi170.front.sepia.ceph.com] => {"changed": true, "cmd": "lsblk --list --noheadings | grep part | grep -v sda | awk '{ print $1 }'", "delta": "0:00:00.005124", "end": "2020-03-19 21:14:08.068062", "rc": 0, "start": "2020-03-19 21:14:08.062938", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-03-19T21:14:08.131 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:08 +0000 (0:00:00.248) 0:02:56.734 ******** 2020-03-19T21:14:08.178 INFO:teuthology.task.ansible.out: TASK [testnode : Unmount any non-root mountpoints] ***************************** 2020-03-19T21:14:08.178 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={u'block_used': 12916307, u'uuid': u'51c8dc47-a34c-4398-9853-098c2389802a', u'size_total': 983429562368, u'block_total': 240095108, u'mount': u'/', u'block_available': 227178801, u'size_available': 930524368896, u'fstype': u'ext4', u'inode_total': 61054976, u'inode_available': 60987191, u'device': u'/dev/sda1', u'inode_used': 67785, u'block_size': 4096, u'options': u'rw,seclabel,relatime'}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 227178801, "block_size": 4096, "block_total": 240095108, "block_used": 12916307, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60987191, "inode_total": 61054976, "inode_used": 67785, "mount": "/", "options": "rw,seclabel,relatime", "size_available": 930524368896, "size_total": 983429562368, "uuid": "51c8dc47-a34c-4398-9853-098c2389802a"}, "skip_reason": "Conditional result was False"} 2020-03-19T21:14:08.221 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:08 +0000 (0:00:00.090) 0:02:56.825 ******** 2020-03-19T21:14:08.249 INFO:teuthology.task.ansible.out: TASK [testnode : Wipe filesystems on non-root partitions] ********************** 2020-03-19T21:14:08.285 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:08 +0000 (0:00:00.063) 0:02:56.889 ******** 2020-03-19T21:14:08.435 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2020-03-19T21:14:08.436 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': u'ATA', u'links': {u'masters': [], u'labels': [], u'ids': [u'ata-ST1000NM0033-9ZM173_Z1W5XK1Y', u'wwn-0x5000c50092940bdb'], u'uuids': []}, u'sas_device_handle': None, u'host': u'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', u'support_discard': u'0', u'serial': u'Z1W5XK1Y', u'wwn': u'0x5000c50092940bdb', u'holders': [], u'size': u'931.51 GB', u'scheduler_mode': u'mq-deadline', u'rotational': u'1', u'sectors': u'1953525168', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': u'ST1000NM0033-9ZM', u'partitions': {u'sda1': {u'start': u'2048', u'sectorsize': 512, u'uuid': u'51c8dc47-a34c-4398-9853-098c2389802a', u'sectors': u'1953522688', u'holders': [], u'links': {u'masters': [], u'labels': [], u'ids': [u'ata-ST1000NM0033-9ZM173_Z1W5XK1Y-part1', u'wwn-0x5000c50092940bdb-part1'], u'uuids': [u'51c8dc47-a34c-4398-9853-098c2389802a']}, u'size': u'931.51 GB'}}}, 'key': u'sda'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5XK1Y", "wwn-0x5000c50092940bdb"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5XK1Y-part1", "wwn-0x5000c50092940bdb-part1"], "labels": [], "masters": [], "uuids": ["51c8dc47-a34c-4398-9853-098c2389802a"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "51c8dc47-a34c-4398-9853-098c2389802a"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W5XK1Y", "size": "931.51 GB", "support_discard": "0", "vend 2020-03-19T21:14:08.437 INFO:teuthology.task.ansible.out:or": "ATA", "virtual": 1, "wwn": "0x5000c50092940bdb"}}, "skip_reason": "Conditional result was False"} 2020-03-19T21:14:10.010 INFO:teuthology.task.ansible.out:changed: [smithi170.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [u'dm-0', u'dm-1', u'dm-2', u'dm-3', u'dm-4'], u'labels': [], u'ids': [u'lvm-pv-uuid-mn7Ip8-9asE-qQct-uT6J-EwvD-PqBj-uXVHSR', u'nvme-INTEL_SSDPEDMD400G4_CVFT6232000C400BGN', u'nvme-nvme.8086-43564654363233323030304334303042474e-494e54454c205353445045444d443430304734-00000001'], u'uuids': []}, u'sas_device_handle': None, u'host': u'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', u'support_discard': u'512', u'model': u'INTEL SSDPEDMD400G4', u'size': u'372.61 GB', u'scheduler_mode': u'none', u'rotational': u'0', u'sectors': u'781422768', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [u'vg_nvme-lv_4', u'vg_nvme-lv_1', u'vg_nvme-lv_3', u'vg_nvme-lv_5', u'vg_nvme-lv_2'], u'partitions': {}}, 'key': u'nvme0n1'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/nvme0n1 || sgdisk --zap-all /dev/nvme0n1", "delta": "0:00:01.385010", "end": "2020-03-19 21:14:09.978809", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_4", "vg_nvme-lv_1", "vg_nvme-lv_3", "vg_nvme-lv_5", "vg_nvme-lv_2"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-mn7Ip8-9asE-qQct-uT6J-EwvD-PqBj-uXVHSR", "nvme-INTEL_SSDPEDMD400G4_CVFT6232000C400BGN", "nvme-nvme.8086-43564654363233323030304334303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "size": "372.61 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-03-19 21:14:08.593799", "stderr": "", "stderr_lines": [], "stdout": "Creating new GP 2020-03-19T21:14:10.011 INFO:teuthology.task.ansible.out:T entries.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-03-19T21:14:11.286 INFO:teuthology.task.ansible.out:changed: [smithi170.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_1', u'dm-uuid-LVM-feeyKMFqLyEH1CFjWs44rhFakZOcK8k0Vf38Cdu5ANa2EkXEi26S4dcfyuK64K0H'], u'uuids': []}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'model': None, u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [], u'partitions': {}}, 'key': u'dm-4'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-4 || sgdisk --zap-all /dev/dm-4", "delta": "0:00:01.053821", "end": "2020-03-19 21:14:11.250228", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-feeyKMFqLyEH1CFjWs44rhFakZOcK8k0Vf38Cdu5ANa2EkXEi26S4dcfyuK64K0H"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-03-19 21:14:10.196407", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-03-19T21:14:12.559 INFO:teuthology.task.ansible.out:changed: [smithi170.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_3', u'dm-uuid-LVM-feeyKMFqLyEH1CFjWs44rhFakZOcK8k0sbZTaVuy3ic7eJk8070afGfGj1EUI40v'], u'uuids': []}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'model': None, u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [], u'partitions': {}}, 'key': u'dm-2'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-2 || sgdisk --zap-all /dev/dm-2", "delta": "0:00:01.065481", "end": "2020-03-19 21:14:12.524794", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-feeyKMFqLyEH1CFjWs44rhFakZOcK8k0sbZTaVuy3ic7eJk8070afGfGj1EUI40v"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-03-19 21:14:11.459313", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-03-19T21:14:13.839 INFO:teuthology.task.ansible.out:changed: [smithi170.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_2', u'dm-uuid-LVM-feeyKMFqLyEH1CFjWs44rhFakZOcK8k0h9lV9tzMAMuAs1KOm3432OmJMFjbz0tq'], u'uuids': []}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'model': None, u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [], u'partitions': {}}, 'key': u'dm-3'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-3 || sgdisk --zap-all /dev/dm-3", "delta": "0:00:01.073624", "end": "2020-03-19 21:14:13.808706", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-feeyKMFqLyEH1CFjWs44rhFakZOcK8k0h9lV9tzMAMuAs1KOm3432OmJMFjbz0tq"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-03-19 21:14:12.735082", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-03-19T21:14:15.093 INFO:teuthology.task.ansible.out:changed: [smithi170.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_5', u'dm-uuid-LVM-feeyKMFqLyEH1CFjWs44rhFakZOcK8k0i0efOGnfmxi4fCVgrlOfIUpSJhBifLbA'], u'uuids': [u'd67bc102-6145-4adb-8810-062b4f35fcab']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'model': None, u'size': u'14.90 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'31252480', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [], u'partitions': {}}, 'key': u'dm-0'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-0 || sgdisk --zap-all /dev/dm-0", "delta": "0:00:01.052235", "end": "2020-03-19 21:14:15.058931", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-feeyKMFqLyEH1CFjWs44rhFakZOcK8k0i0efOGnfmxi4fCVgrlOfIUpSJhBifLbA"], "labels": [], "masters": [], "uuids": ["d67bc102-6145-4adb-8810-062b4f35fcab"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-03-19 21:14:14.006696", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-03-19T21:14:16.465 INFO:teuthology.task.ansible.out:changed: [smithi170.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_4', u'dm-uuid-LVM-feeyKMFqLyEH1CFjWs44rhFakZOcK8k0Q8YFqCc3627cw5Rh55YQDHJzmNoOrIyf'], u'uuids': []}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'model': None, u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [], u'partitions': {}}, 'key': u'dm-1'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-1 || sgdisk --zap-all /dev/dm-1", "delta": "0:00:01.080619", "end": "2020-03-19 21:14:16.342294", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-feeyKMFqLyEH1CFjWs44rhFakZOcK8k0Q8YFqCc3627cw5Rh55YQDHJzmNoOrIyf"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-03-19 21:14:15.261675", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-03-19T21:14:16.517 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:16 +0000 (0:00:08.232) 0:03:05.121 ******** 2020-03-19T21:14:16.565 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2020-03-19T21:14:16.566 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': u'ATA', u'links': {u'masters': [], u'labels': [], u'ids': [u'ata-ST1000NM0033-9ZM173_Z1W5XK1Y', u'wwn-0x5000c50092940bdb'], u'uuids': []}, u'sas_device_handle': None, u'host': u'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', u'support_discard': u'0', u'serial': u'Z1W5XK1Y', u'wwn': u'0x5000c50092940bdb', u'holders': [], u'size': u'931.51 GB', u'scheduler_mode': u'mq-deadline', u'rotational': u'1', u'sectors': u'1953525168', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': u'ST1000NM0033-9ZM', u'partitions': {u'sda1': {u'start': u'2048', u'sectorsize': 512, u'uuid': u'51c8dc47-a34c-4398-9853-098c2389802a', u'sectors': u'1953522688', u'holders': [], u'links': {u'masters': [], u'labels': [], u'ids': [u'ata-ST1000NM0033-9ZM173_Z1W5XK1Y-part1', u'wwn-0x5000c50092940bdb-part1'], u'uuids': [u'51c8dc47-a34c-4398-9853-098c2389802a']}, u'size': u'931.51 GB'}}}, 'key': u'sda'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5XK1Y", "wwn-0x5000c50092940bdb"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5XK1Y-part1", "wwn-0x5000c50092940bdb-part1"], "labels": [], "masters": [], "uuids": ["51c8dc47-a34c-4398-9853-098c2389802a"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "51c8dc47-a34c-4398-9853-098c2389802a"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W5XK1Y", "size": "931.51 GB", "support_discard": "0", "vend 2020-03-19T21:14:16.567 INFO:teuthology.task.ansible.out:or": "ATA", "virtual": 1, "wwn": "0x5000c50092940bdb"}}, "skip_reason": "Conditional result was False"} 2020-03-19T21:14:16.805 INFO:teuthology.task.ansible.out:changed: [smithi170.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [u'dm-0', u'dm-1', u'dm-2', u'dm-3', u'dm-4'], u'labels': [], u'ids': [u'lvm-pv-uuid-mn7Ip8-9asE-qQct-uT6J-EwvD-PqBj-uXVHSR', u'nvme-INTEL_SSDPEDMD400G4_CVFT6232000C400BGN', u'nvme-nvme.8086-43564654363233323030304334303042474e-494e54454c205353445045444d443430304734-00000001'], u'uuids': []}, u'sas_device_handle': None, u'host': u'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', u'support_discard': u'512', u'model': u'INTEL SSDPEDMD400G4', u'size': u'372.61 GB', u'scheduler_mode': u'none', u'rotational': u'0', u'sectors': u'781422768', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [u'vg_nvme-lv_4', u'vg_nvme-lv_1', u'vg_nvme-lv_3', u'vg_nvme-lv_5', u'vg_nvme-lv_2'], u'partitions': {}}, 'key': u'nvme0n1'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/nvme0n1 bs=1M count=110", "delta": "0:00:00.037521", "end": "2020-03-19 21:14:16.775620", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_4", "vg_nvme-lv_1", "vg_nvme-lv_3", "vg_nvme-lv_5", "vg_nvme-lv_2"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-mn7Ip8-9asE-qQct-uT6J-EwvD-PqBj-uXVHSR", "nvme-INTEL_SSDPEDMD400G4_CVFT6232000C400BGN", "nvme-nvme.8086-43564654363233323030304334303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "size": "372.61 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-03-19 21:14:16.738099", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 11 2020-03-19T21:14:16.807 INFO:teuthology.task.ansible.out:0 MiB) copied, 0.034837 s, 3.3 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.034837 s, 3.3 GB/s"], "stdout": "", "stdout_lines": []} 2020-03-19T21:14:17.140 INFO:teuthology.task.ansible.out:changed: [smithi170.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_1', u'dm-uuid-LVM-feeyKMFqLyEH1CFjWs44rhFakZOcK8k0Vf38Cdu5ANa2EkXEi26S4dcfyuK64K0H'], u'uuids': []}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'model': None, u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [], u'partitions': {}}, 'key': u'dm-4'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-4 bs=1M count=110", "delta": "0:00:00.140088", "end": "2020-03-19 21:14:17.118404", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-feeyKMFqLyEH1CFjWs44rhFakZOcK8k0Vf38Cdu5ANa2EkXEi26S4dcfyuK64K0H"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-03-19 21:14:16.978316", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.137454 s, 839 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.137454 s, 839 MB/s"], "stdout": "", "stdout_lines": []} 2020-03-19T21:14:17.467 INFO:teuthology.task.ansible.out:changed: [smithi170.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_3', u'dm-uuid-LVM-feeyKMFqLyEH1CFjWs44rhFakZOcK8k0sbZTaVuy3ic7eJk8070afGfGj1EUI40v'], u'uuids': []}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'model': None, u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [], u'partitions': {}}, 'key': u'dm-2'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-2 bs=1M count=110", "delta": "0:00:00.139416", "end": "2020-03-19 21:14:17.440944", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-feeyKMFqLyEH1CFjWs44rhFakZOcK8k0sbZTaVuy3ic7eJk8070afGfGj1EUI40v"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-03-19 21:14:17.301528", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.13685 s, 843 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.13685 s, 843 MB/s"], "stdout": "", "stdout_lines": []} 2020-03-19T21:14:17.821 INFO:teuthology.task.ansible.out:changed: [smithi170.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_2', u'dm-uuid-LVM-feeyKMFqLyEH1CFjWs44rhFakZOcK8k0h9lV9tzMAMuAs1KOm3432OmJMFjbz0tq'], u'uuids': []}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'model': None, u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [], u'partitions': {}}, 'key': u'dm-3'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-3 bs=1M count=110", "delta": "0:00:00.150223", "end": "2020-03-19 21:14:17.788430", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-feeyKMFqLyEH1CFjWs44rhFakZOcK8k0h9lV9tzMAMuAs1KOm3432OmJMFjbz0tq"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-03-19 21:14:17.638207", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.147439 s, 782 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.147439 s, 782 MB/s"], "stdout": "", "stdout_lines": []} 2020-03-19T21:14:18.154 INFO:teuthology.task.ansible.out:changed: [smithi170.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_5', u'dm-uuid-LVM-feeyKMFqLyEH1CFjWs44rhFakZOcK8k0i0efOGnfmxi4fCVgrlOfIUpSJhBifLbA'], u'uuids': [u'd67bc102-6145-4adb-8810-062b4f35fcab']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'model': None, u'size': u'14.90 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'31252480', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [], u'partitions': {}}, 'key': u'dm-0'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-0 bs=1M count=110", "delta": "0:00:00.152850", "end": "2020-03-19 21:14:18.123463", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-feeyKMFqLyEH1CFjWs44rhFakZOcK8k0i0efOGnfmxi4fCVgrlOfIUpSJhBifLbA"], "labels": [], "masters": [], "uuids": ["d67bc102-6145-4adb-8810-062b4f35fcab"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-03-19 21:14:17.970613", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.15022 s, 768 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.15022 s, 768 MB/s"], "stdout": "", "stdout_lines": []} 2020-03-19T21:14:18.475 INFO:teuthology.task.ansible.out:changed: [smithi170.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_4', u'dm-uuid-LVM-feeyKMFqLyEH1CFjWs44rhFakZOcK8k0Q8YFqCc3627cw5Rh55YQDHJzmNoOrIyf'], u'uuids': []}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'model': None, u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [], u'partitions': {}}, 'key': u'dm-1'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-1 bs=1M count=110", "delta": "0:00:00.139122", "end": "2020-03-19 21:14:18.445957", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-feeyKMFqLyEH1CFjWs44rhFakZOcK8k0Q8YFqCc3627cw5Rh55YQDHJzmNoOrIyf"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-03-19 21:14:18.306835", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.136518 s, 845 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.136518 s, 845 MB/s"], "stdout": "", "stdout_lines": []} 2020-03-19T21:14:18.533 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:18 +0000 (0:00:02.015) 0:03:07.136 ******** 2020-03-19T21:14:18.796 INFO:teuthology.task.ansible.out: TASK [testnode : Remove all LVM data] ****************************************** 2020-03-19T21:14:18.798 INFO:teuthology.task.ansible.out:changed: [smithi170.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "cmd": "dmsetup remove_all --force", "delta": "0:00:00.062955", "end": "2020-03-19 21:14:18.766128", "rc": 0, "start": "2020-03-19 21:14:18.703173", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-03-19T21:14:18.834 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:18 +0000 (0:00:00.300) 0:03:07.437 ******** 2020-03-19T21:14:19.077 INFO:teuthology.task.ansible.out: TASK [testnode : Check for physical volumes] *********************************** 2020-03-19T21:14:19.078 INFO:teuthology.task.ansible.out:changed: [smithi170.front.sepia.ceph.com] => {"changed": true, "cmd": "pvdisplay | grep 'PV Name' | awk '{ print $3 }'", "delta": "0:00:00.048475", "end": "2020-03-19 21:14:19.048996", "rc": 0, "start": "2020-03-19 21:14:19.000521", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-03-19T21:14:19.121 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:19 +0000 (0:00:00.287) 0:03:07.724 ******** 2020-03-19T21:14:19.147 INFO:teuthology.task.ansible.out: TASK [testnode : Remove physical volumes] ************************************** 2020-03-19T21:14:19.194 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:19 +0000 (0:00:00.072) 0:03:07.797 ******** 2020-03-19T21:14:19.221 INFO:teuthology.task.ansible.out: TASK [testnode : Write a new partition table to {{ item.value.device }}] ******* 2020-03-19T21:14:19.222 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:14:19.264 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:19 +0000 (0:00:00.069) 0:03:07.867 ******** 2020-03-19T21:14:19.290 INFO:teuthology.task.ansible.out: TASK [testnode : Write partition entries to {{ item.0.device }}] *************** 2020-03-19T21:14:19.292 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:14:19.334 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:19 +0000 (0:00:00.070) 0:03:07.937 ******** 2020-03-19T21:14:19.360 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2020-03-19T21:14:19.361 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:14:19.402 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:19 +0000 (0:00:00.067) 0:03:08.005 ******** 2020-03-19T21:14:19.428 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs for {{ item.0.device }}] ****************** 2020-03-19T21:14:19.430 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:14:19.469 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:19 +0000 (0:00:00.066) 0:03:08.072 ******** 2020-03-19T21:14:19.665 INFO:teuthology.task.ansible.out: TASK [testnode : Edit lvm.conf to support LVM on kRBD.] ************************ 2020-03-19T21:14:19.666 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-03-19T21:14:19.731 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:19 +0000 (0:00:00.262) 0:03:08.335 ******** 2020-03-19T21:14:19.767 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2020-03-19T21:14:19.768 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={u'block_used': 12916307, u'uuid': u'51c8dc47-a34c-4398-9853-098c2389802a', u'size_total': 983429562368, u'block_total': 240095108, u'mount': u'/', u'block_available': 227178801, u'size_available': 930524368896, u'fstype': u'ext4', u'inode_total': 61054976, u'inode_available': 60987191, u'device': u'/dev/sda1', u'inode_used': 67785, u'block_size': 4096, u'options': u'rw,seclabel,relatime'}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 227178801, "block_size": 4096, "block_total": 240095108, "block_used": 12916307, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60987191, "inode_total": 61054976, "inode_used": 67785, "mount": "/", "options": "rw,seclabel,relatime", "size_available": 930524368896, "size_total": 983429562368, "uuid": "51c8dc47-a34c-4398-9853-098c2389802a"}, "skip_reason": "Conditional result was False"} 2020-03-19T21:14:19.809 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:19 +0000 (0:00:00.077) 0:03:08.412 ******** 2020-03-19T21:14:19.834 INFO:teuthology.task.ansible.out: TASK [testnode : Combine list of non-root disks] ******************************* 2020-03-19T21:14:19.835 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:14:19.870 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:19 +0000 (0:00:00.060) 0:03:08.473 ******** 2020-03-19T21:14:19.895 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-03-19T21:14:19.896 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:14:19.931 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:19 +0000 (0:00:00.061) 0:03:08.534 ******** 2020-03-19T21:14:19.977 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-03-19T21:14:19.979 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:14:20.022 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:20 +0000 (0:00:00.091) 0:03:08.625 ******** 2020-03-19T21:14:20.049 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume_groups dict] ************************************ 2020-03-19T21:14:20.050 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:14:20.085 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:20 +0000 (0:00:00.063) 0:03:08.688 ******** 2020-03-19T21:14:20.111 INFO:teuthology.task.ansible.out: TASK [testnode : Determine desired logical volume percentage size] ************* skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:14:20.147 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:20 +0000 (0:00:00.062) 0:03:08.750 ******** 2020-03-19T21:14:20.173 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical_volumes dict] ********************************** 2020-03-19T21:14:20.175 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:14:20.210 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:20 +0000 (0:00:00.062) 0:03:08.813 ******** 2020-03-19T21:14:20.832 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume group(s)] *************************************** 2020-03-19T21:14:20.834 INFO:teuthology.task.ansible.out:changed: [smithi170.front.sepia.ceph.com] => (item={'value': {u'pvs': u'/dev/nvme0n1'}, 'key': u'vg_nvme'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "vg_nvme", "value": {"pvs": "/dev/nvme0n1"}}} 2020-03-19T21:14:20.869 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:20 +0000 (0:00:00.659) 0:03:09.473 ******** 2020-03-19T21:14:21.409 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2020-03-19T21:14:21.410 INFO:teuthology.task.ansible.out:changed: [smithi170.front.sepia.ceph.com] => (item={'value': {u'vg': u'vg_nvme', u'size': u'14.9G'}, 'key': u'lv_5'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "msg": ""} 2020-03-19T21:14:21.738 INFO:teuthology.task.ansible.out:changed: [smithi170.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_4'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-03-19T21:14:22.084 INFO:teuthology.task.ansible.out:changed: [smithi170.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_3'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-03-19T21:14:22.421 INFO:teuthology.task.ansible.out:changed: [smithi170.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_2'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-03-19T21:14:22.781 INFO:teuthology.task.ansible.out:changed: [smithi170.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_1'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-03-19T21:14:22.822 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:22 +0000 (0:00:01.952) 0:03:11.426 ******** 2020-03-19T21:14:23.014 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2020-03-19T21:14:23.016 INFO:teuthology.task.ansible.out:changed: [smithi170.front.sepia.ceph.com] => {"changed": true, "path": "/scratch_devs", "state": "absent"} 2020-03-19T21:14:23.052 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:23 +0000 (0:00:00.229) 0:03:11.656 ******** 2020-03-19T21:14:23.099 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2020-03-19T21:14:23.101 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'value': {u'vg': u'vg_nvme', u'size': u'14.9G'}, 'key': u'lv_5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "skip_reason": "Conditional result was False"} 2020-03-19T21:14:23.286 INFO:teuthology.task.ansible.out:changed: [smithi170.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_4'}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-03-19T21:14:23.493 INFO:teuthology.task.ansible.out:changed: [smithi170.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_3'}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-03-19T21:14:23.691 INFO:teuthology.task.ansible.out:changed: [smithi170.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_2'}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-03-19T21:14:23.877 INFO:teuthology.task.ansible.out:changed: [smithi170.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_1'}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-03-19T21:14:23.924 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:23 +0000 (0:00:00.871) 0:03:12.527 ******** 2020-03-19T21:14:24.125 INFO:teuthology.task.ansible.out: TASK [testnode : Create /var/lib/ceph] ***************************************** 2020-03-19T21:14:24.126 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/var/lib/ceph", "secontext": "unconfined_u:object_r:var_lib_t:s0", "size": 4096, "state": "directory", "uid": 0} 2020-03-19T21:14:24.166 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:24 +0000 (0:00:00.242) 0:03:12.769 ******** 2020-03-19T21:14:24.191 INFO:teuthology.task.ansible.out: TASK [testnode : Set xfs_opts on newer OSes] *********************************** 2020-03-19T21:14:24.192 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:14:24.232 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:24 +0000 (0:00:00.065) 0:03:12.835 ******** 2020-03-19T21:14:24.671 INFO:teuthology.task.ansible.out: TASK [testnode : Create xfs filesystem on /dev/vg_nvme/lv_5] ******************* 2020-03-19T21:14:24.672 INFO:teuthology.task.ansible.out:changed: [smithi170.front.sepia.ceph.com] => {"changed": true} 2020-03-19T21:14:24.711 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:24 +0000 (0:00:00.478) 0:03:13.314 ******** 2020-03-19T21:14:25.175 INFO:teuthology.task.ansible.out: TASK [testnode : Mount /dev/vg_nvme/lv_5 to /var/lib/ceph] ********************* 2020-03-19T21:14:25.177 INFO:teuthology.task.ansible.out:changed: [smithi170.front.sepia.ceph.com] => {"changed": true, "dump": "0", "fstab": "/etc/fstab", "fstype": "xfs", "name": "/var/lib/ceph", "opts": "defaults,nofail", "passno": "0", "src": "/dev/vg_nvme/lv_5"} 2020-03-19T21:14:25.213 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:25 +0000 (0:00:00.502) 0:03:13.817 ******** 2020-03-19T21:14:25.238 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the teuthology user.] *********** 2020-03-19T21:14:25.240 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:14:25.279 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:25 +0000 (0:00:00.065) 0:03:13.882 ******** 2020-03-19T21:14:25.304 INFO:teuthology.task.ansible.out: 2020-03-19T21:14:25.306 INFO:teuthology.task.ansible.out:TASK [testnode : Add CPAN config directory for the root user.] ***************** 2020-03-19T21:14:25.309 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:14:25.349 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:25 +0000 (0:00:00.069) 0:03:13.952 ******** 2020-03-19T21:14:25.350 INFO:teuthology.task.ansible.out: 2020-03-19T21:14:25.375 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for the teuthology user.] ****************** 2020-03-19T21:14:25.377 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:14:25.420 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:25 +0000 (0:00:00.071) 0:03:14.023 ******** 2020-03-19T21:14:25.453 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for root.] ********************************* skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:14:25.492 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:25 +0000 (0:00:00.071) 0:03:14.095 ******** 2020-03-19T21:14:25.518 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure perl-doc is installed on apt systems.] ***************** 2020-03-19T21:14:25.520 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:14:25.560 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:25 +0000 (0:00:00.068) 0:03:14.163 ******** 2020-03-19T21:14:25.586 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if Amazon::S3 is installed.] ********************* 2020-03-19T21:14:25.587 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:14:25.625 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:25 +0000 (0:00:00.065) 0:03:14.229 ******** 2020-03-19T21:14:25.652 INFO:teuthology.task.ansible.out: TASK [testnode : Install Amazon::S3.] ****************************************** 2020-03-19T21:14:25.653 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:14:25.689 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:25 +0000 (0:00:00.063) 0:03:14.292 ******** 2020-03-19T21:14:25.714 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on rpm based systems.] ******************** 2020-03-19T21:14:25.716 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:14:25.750 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:25 +0000 (0:00:00.061) 0:03:14.353 ******** 2020-03-19T21:14:25.775 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on apt based systems.] ******************** 2020-03-19T21:14:25.776 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:14:25.820 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:25 +0000 (0:00:00.069) 0:03:14.423 ******** 2020-03-19T21:14:25.821 INFO:teuthology.task.ansible.out: 2020-03-19T21:14:25.855 INFO:teuthology.task.ansible.out: TASK [testnode : Remove racy /etc/network/if-up.d/ntpdate on xenial] *********** 2020-03-19T21:14:25.857 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:14:25.893 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:25 +0000 (0:00:00.073) 0:03:14.496 ******** 2020-03-19T21:14:25.919 INFO:teuthology.task.ansible.out: TASK [testnode : Create the ntp.conf file.] ************************************ 2020-03-19T21:14:25.920 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:14:25.955 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:25 +0000 (0:00:00.061) 0:03:14.558 ******** 2020-03-19T21:14:26.416 INFO:teuthology.task.ansible.out: TASK [testnode : Create the chrony.conf file] ********************************** 2020-03-19T21:14:26.416 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => {"changed": false, "checksum": "a7ecef00073feab34f868065d34e9dac55134241", "dest": "/etc/chrony.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/chrony.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 215, "state": "file", "uid": 0} 2020-03-19T21:14:26.454 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:26 +0000 (0:00:00.498) 0:03:15.057 ******** 2020-03-19T21:14:26.759 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ntpd is running.] *********************************** 2020-03-19T21:14:26.761 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "chronyd", "state": "started", "status": {"ActiveEnterTimestamp": "Thu 2020-03-19 21:10:20 UTC", "ActiveEnterTimestampMonotonic": "15240210", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "systemd-journald.socket sntp.service ntpdate.service basic.target systemd-tmpfiles-setup.service ntpd.service system.slice -.mount tmp.mount sysinit.target", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Thu 2020-03-19 21:10:19 UTC", "AssertTimestampMonotonic": "13601391", "Before": "multi-user.target shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "no", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Thu 2020-03-19 21:10:19 UTC", "ConditionTimestampMonotonic": "13601365", "ConfigurationDirectoryMode": "0755", "Conflicts": "shutdown.target ntpd.service systemd-timesyncd.service", "ControlGroup": "/system.slice/chronyd.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description" 2020-03-19T21:14:26.762 INFO:teuthology.task.ansible.out:: "NTP client/server", "DevicePolicy": "auto", "Documentation": "man:chronyd(8) man:chrony.conf(5)", "DynamicUser": "no", "EnvironmentFiles": "/etc/sysconfig/chronyd (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "2370", "ExecMainStartTimestamp": "Thu 2020-03-19 21:10:20 UTC", "ExecMainStartTimestampMonotonic": "15008026", "ExecMainStatus": "0", "ExecStart": "{ path=/usr/sbin/chronyd ; argv[]=/usr/sbin/chronyd $OPTIONS ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStartPost": "{ path=/usr/libexec/chrony-helper ; argv[]=/usr/libexec/chrony-helper update-daemon ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/chronyd.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "chronyd.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Thu 2020-03-19 21:10:19 UTC", "InactiveExitTimestampMonotonic": "13679414", "InvocationID": "9aea248737a2497788c913caa28cc6f2", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "infinity", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "167772 2020-03-19T21:14:26.763 INFO:teuthology.task.ansible.out:16", "LimitMEMLOCKSoft": "16777216", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "127442", "LimitNPROCSoft": "127442", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "127442", "LimitSIGPENDINGSoft": "127442", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "2370", "MemoryAccounting": "yes", "MemoryCurrent": "2617344", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "chronyd.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PIDFile": "/run/chrony/chronyd.pid", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateMounts": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "yes", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "full", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "-.mount sysinit.target system.slice", "RequiresMountsFor": "/var/tmp", "Restart": "no", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardE 2020-03-19T21:14:26.765 INFO:teuthology.task.ansible.out:rror": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Thu 2020-03-19 21:10:20 UTC", "StateChangeTimestampMonotonic": "15240210", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "1", "TasksMax": "26213", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "forking", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Thu 2020-03-19 21:10:20 UTC", "WatchdogTimestampMonotonic": "15008040", "WatchdogUSec": "0"}} 2020-03-19T21:14:26.800 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:26 +0000 (0:00:00.346) 0:03:15.404 ******** 2020-03-19T21:14:26.825 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on rpm based systems.] ********************* 2020-03-19T21:14:26.827 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:14:26.861 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:26 +0000 (0:00:00.060) 0:03:15.464 ******** 2020-03-19T21:14:26.889 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on apt based systems.] ********************* skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:14:26.922 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:26 +0000 (0:00:00.061) 0:03:15.526 ******** 2020-03-19T21:14:26.948 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on zypper based systems.] ****************** 2020-03-19T21:14:26.949 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:14:26.986 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:26 +0000 (0:00:00.062) 0:03:15.588 ******** 2020-03-19T21:14:27.184 INFO:teuthology.task.ansible.out: TASK [testnode : Create the .pip directory for the teuthology user.] *********** 2020-03-19T21:14:27.185 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => {"changed": false, "gid": 1211, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 4096, "state": "directory", "uid": 1000} 2020-03-19T21:14:27.223 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:27 +0000 (0:00:00.236) 0:03:15.825 ******** 2020-03-19T21:14:27.692 INFO:teuthology.task.ansible.out: TASK [testnode : Create pip.conf and configure it to use our mirror] *********** 2020-03-19T21:14:27.692 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => {"changed": false, "checksum": "959d37ab30545da205b389a6b7561b225a478a06", "dest": "/home/ubuntu/.pip/pip.conf", "gid": 1211, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip/pip.conf", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 73, "state": "file", "uid": 1000} 2020-03-19T21:14:27.729 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:27 +0000 (0:00:00.507) 0:03:16.332 ******** 2020-03-19T21:14:28.901 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages via pip] ************************************* 2020-03-19T21:14:28.902 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => {"changed": false, "cmd": ["/bin/pip3", "install", "remoto>=0.0.35"], "name": ["remoto>=0.0.35"], "requirements": null, "state": "present", "stderr": "WARNING: Running pip install with root privileges is generally not a good idea. Try `pip3 install --user` instead.\n", "stderr_lines": ["WARNING: Running pip install with root privileges is generally not a good idea. Try `pip3 install --user` instead."], "stdout": "Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python3.6/site-packages\nRequirement already satisfied: execnet in /usr/local/lib/python3.6/site-packages (from remoto>=0.0.35)\nRequirement already satisfied: apipkg>=1.4 in /usr/local/lib/python3.6/site-packages (from execnet->remoto>=0.0.35)\n", "stdout_lines": ["Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python3.6/site-packages", "Requirement already satisfied: execnet in /usr/local/lib/python3.6/site-packages (from remoto>=0.0.35)", "Requirement already satisfied: apipkg>=1.4 in /usr/local/lib/python3.6/site-packages (from execnet->remoto>=0.0.35)"], "version": null, "virtualenv": null} 2020-03-19T21:14:28.947 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:28 +0000 (0:00:01.217) 0:03:17.550 ******** 2020-03-19T21:14:28.973 INFO:teuthology.task.ansible.out: TASK [testnode : Purge resolvconf] ********************************************* 2020-03-19T21:14:28.975 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:14:29.012 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:29 +0000 (0:00:00.064) 0:03:17.615 ******** 2020-03-19T21:14:29.038 INFO:teuthology.task.ansible.out: TASK [testnode : Set interface] ************************************************ 2020-03-19T21:14:29.039 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => {"ansible_facts": {"interface": "enp3s0f1"}, "changed": false} 2020-03-19T21:14:29.078 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:29 +0000 (0:00:00.066) 0:03:17.681 ******** 2020-03-19T21:14:29.272 INFO:teuthology.task.ansible.out: TASK [testnode : Check for presence of /etc/network/interfaces] **************** 2020-03-19T21:14:29.273 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => {"changed": false, "stat": {"exists": false}} 2020-03-19T21:14:29.311 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:29 +0000 (0:00:00.233) 0:03:17.915 ******** 2020-03-19T21:14:29.337 INFO:teuthology.task.ansible.out: TASK [testnode : Rewrite /etc/network/interfaces to use dhcp] ****************** 2020-03-19T21:14:29.340 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:14:29.381 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:29 +0000 (0:00:00.064) 0:03:17.979 ******** 2020-03-19T21:14:29.414 INFO:teuthology.task.ansible.out: TASK [testnode : Set bounce_interface if we just enabled dhcp] ***************** 2020-03-19T21:14:29.415 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => {"ansible_facts": {"bounce_interface": false}, "changed": false} 2020-03-19T21:14:29.452 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:29 +0000 (0:00:00.075) 0:03:18.055 ******** 2020-03-19T21:14:29.476 INFO:teuthology.task.ansible.out: TASK [testnode : ifdown and ifup] ********************************************** 2020-03-19T21:14:29.478 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:14:29.516 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:29 +0000 (0:00:00.063) 0:03:18.119 ******** 2020-03-19T21:14:29.718 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure lab_domain is in search domains in /etc/resolv.conf] *** 2020-03-19T21:14:29.720 INFO:teuthology.task.ansible.out:changed: [smithi170.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line replaced"} 2020-03-19T21:14:29.756 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:29 +0000 (0:00:00.240) 0:03:18.359 ******** 2020-03-19T21:14:29.952 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure domain is set in /etc/resolv.conf] ********************* changed: [smithi170.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line added"} 2020-03-19T21:14:29.991 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:29 +0000 (0:00:00.234) 0:03:18.594 ******** 2020-03-19T21:14:30.017 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ethtool is installed] ******************************* 2020-03-19T21:14:30.019 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:14:30.056 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:30 +0000 (0:00:00.065) 0:03:18.659 ******** 2020-03-19T21:14:30.094 INFO:teuthology.task.ansible.out: TASK [testnode : grep ethtool for secondary NIC MAC address] ******************* 2020-03-19T21:14:30.095 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item=lo) => {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False"} 2020-03-19T21:14:30.097 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item=enp3s0f1) => {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f1", "skip_reason": "Conditional result was False"} 2020-03-19T21:14:30.106 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item=eno2) => {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False"} 2020-03-19T21:14:30.117 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item=eno1) => {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False"} 2020-03-19T21:14:30.118 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item=enp3s0f0) => {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f0", "skip_reason": "Conditional result was False"} 2020-03-19T21:14:30.159 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:30 +0000 (0:00:00.102) 0:03:18.762 ******** 2020-03-19T21:14:30.195 INFO:teuthology.task.ansible.out: TASK [testnode : Define net_to_configure var] ********************************** 2020-03-19T21:14:30.196 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'skip_reason': u'Conditional result was False', 'item': u'lo', 'skipped': True, 'ansible_loop_var': u'item', 'changed': False}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-03-19T21:14:30.197 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'skip_reason': u'Conditional result was False', 'item': u'enp3s0f1', 'skipped': True, 'ansible_loop_var': u'item', 'changed': False}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-03-19T21:14:30.209 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'skip_reason': u'Conditional result was False', 'item': u'eno2', 'skipped': True, 'ansible_loop_var': u'item', 'changed': False}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-03-19T21:14:30.211 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'skip_reason': u'Conditional result was False', 'item': u'eno1', 'skipped': True, 'ansible_loop_var': u'item', 'changed': False}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-03-19T21:14:30.222 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => (item={'skip_reason': u'Conditional result was False', 'item': u'enp3s0f0', 'skipped': True, 'ansible_loop_var': u'item', 'changed': False}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f0", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-03-19T21:14:30.260 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:30 +0000 (0:00:00.100) 0:03:18.863 ******** 2020-03-19T21:14:30.286 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-03-19T21:14:30.288 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:14:30.338 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:30 +0000 (0:00:00.078) 0:03:18.941 ******** 2020-03-19T21:14:30.363 INFO:teuthology.task.ansible.out: TASK [testnode : Check if {{ nic_to_configure }} is 10Gb] ********************** 2020-03-19T21:14:30.365 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:14:30.399 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:30 +0000 (0:00:00.060) 0:03:19.002 ******** 2020-03-19T21:14:30.424 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU to 9000 if 10Gb] ************************************** 2020-03-19T21:14:30.426 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:14:30.462 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:30 +0000 (0:00:00.063) 0:03:19.066 ******** 2020-03-19T21:14:30.487 INFO:teuthology.task.ansible.out: TASK [testnode : Write Ubuntu network config for {{ nic_to_configure }}] ******* 2020-03-19T21:14:30.489 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:14:30.523 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:30 +0000 (0:00:00.060) 0:03:19.126 ******** 2020-03-19T21:14:30.548 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU on Ubuntu] ******************************************** 2020-03-19T21:14:30.549 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:14:30.586 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:30 +0000 (0:00:00.062) 0:03:19.189 ******** 2020-03-19T21:14:30.611 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2020-03-19T21:14:30.612 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:14:30.648 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:30 +0000 (0:00:00.062) 0:03:19.252 ******** 2020-03-19T21:14:30.674 INFO:teuthology.task.ansible.out: TASK [testnode : Write RHEL/CentOS network config for {{ nic_to_configure }}] *** 2020-03-19T21:14:30.675 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:14:30.711 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:30 +0000 (0:00:00.062) 0:03:19.315 ******** 2020-03-19T21:14:30.736 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2020-03-19T21:14:30.737 INFO:teuthology.task.ansible.out:skipping: [smithi170.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T21:14:30.772 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:30 +0000 (0:00:00.060) 0:03:19.375 ******** 2020-03-19T21:14:31.002 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2020-03-19T21:14:31.004 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item=/var/run/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/run/ -name \"*ceph*\"", "delta": "0:00:00.005192", "end": "2020-03-19 21:14:30.965040", "failed_when_result": false, "item": "/var/run/", "rc": 0, "start": "2020-03-19 21:14:30.959848", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-03-19T21:14:31.184 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item=/etc/systemd/system/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/systemd/system/ -name \"*ceph*\"", "delta": "0:00:00.003032", "end": "2020-03-19 21:14:31.151143", "failed_when_result": false, "item": "/etc/systemd/system/", "rc": 0, "start": "2020-03-19 21:14:31.148111", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-03-19T21:14:31.367 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item=/etc/ceph) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/ceph -name \"*ceph*\"", "delta": "0:00:00.002868", "end": "2020-03-19 21:14:31.336098", "failed_when_result": false, "item": "/etc/ceph", "msg": "non-zero return code", "rc": 1, "start": "2020-03-19 21:14:31.333230", "stderr": "find: ‘/etc/ceph’: No such file or directory", "stderr_lines": ["find: ‘/etc/ceph’: No such file or directory"], "stdout": "", "stdout_lines": []} 2020-03-19T21:14:31.564 INFO:teuthology.task.ansible.out:ok: [smithi170.front.sepia.ceph.com] => (item=/var/log/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/log/ -name \"*ceph*\"", "delta": "0:00:00.014111", "end": "2020-03-19 21:14:31.530240", "failed_when_result": false, "item": "/var/log/", "rc": 0, "start": "2020-03-19 21:14:31.516129", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-03-19T21:14:31.603 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:31 +0000 (0:00:00.831) 0:03:20.206 ******** 2020-03-19T21:14:31.639 INFO:teuthology.task.ansible.out: TASK [testnode : Delete any leftover Ceph artifacts from previous jobs] ******** 2020-03-19T21:14:31.674 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 21:14:31 +0000 (0:00:00.070) 0:03:20.277 ******** 2020-03-19T21:14:31.867 INFO:teuthology.task.ansible.out: TASK [testnode : Touch /ceph-qa-ready] ***************************************** 2020-03-19T21:14:31.868 INFO:teuthology.task.ansible.out:changed: [smithi170.front.sepia.ceph.com] => {"changed": true, "dest": "/ceph-qa-ready", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "secontext": "unconfined_u:object_r:etc_runtime_t:s0", "size": 0, "state": "file", "uid": 0} 2020-03-19T21:14:31.967 INFO:teuthology.task.ansible.out: PLAY [cobbler] ***************************************************************** skipping: no hosts matched 2020-03-19T21:14:31.969 INFO:teuthology.task.ansible.out: PLAY [paddles] ***************************************************************** skipping: no hosts matched 2020-03-19T21:14:31.971 INFO:teuthology.task.ansible.out: PLAY [pulpito] ***************************************************************** skipping: no hosts matched 2020-03-19T21:14:31.972 INFO:teuthology.task.ansible.out: PLAY RECAP ********************************************************************* smithi170.front.sepia.ceph.com : ok=121 changed=35 unreachable=0 failed=0 skipped=162 rescued=0 ignored=4 Thursday 19 March 2020 21:14:31 +0000 (0:00:00.297) 0:03:20.575 ******** =============================================================================== common : nrpe - Load SELinux policy package ---------------------------- 16.63s common : nrpe - Remove SELinux policy package -------------------------- 16.45s testnode : Install packages -------------------------------------------- 13.39s common : Enable necessary rhsm repos ----------------------------------- 12.46s testnode : Base group workaround for BZ1782899 ------------------------- 11.63s common : Get list of enabled RHSM repos --------------------------------- 9.72s common : Register with subscription-manager. ---------------------------- 9.67s testnode : Zap all non-root disks --------------------------------------- 8.23s 2020-03-19T21:14:31.973 INFO:teuthology.task.ansible.out:testnode : Ensure ceph dependency packages are not present. ------------- 7.09s testnode : Ensure ceph packages are not present. ------------------------ 6.83s common : Disable all rhsm repos ----------------------------------------- 5.61s Gathering Facts --------------------------------------------------------- 2.84s common : Install Kerberos Packages (RedHat) ----------------------------- 2.17s testnode : Install epel packages ---------------------------------------- 2.15s testnode : Make sure rpm dependencies are installed --------------------- 2.05s testnode : Remove packages ---------------------------------------------- 2.04s testnode : Blow away lingering OSD data and FSIDs ----------------------- 2.02s common : Install nrpe package and dependencies (RHEL/CentOS) ------------ 2.00s common : nrpe - Install SELinux dependencies ---------------------------- 2.00s common : nrpe - Ensure SELinux policy is up to date --------------------- 1.96s 2020-03-19T21:14:32.237 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi170.front.sepia.ceph.com')] 2020-03-19T21:14:32.239 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-03-19T21:14:32.240 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi170.front.sepia.ceph.com', 'timeout': 60} 2020-03-19T21:14:32.308 INFO:teuthology.orchestra.run.smithi170:> true 2020-03-19T21:14:32.393 INFO:teuthology.run_tasks:Running task clock... 2020-03-19T21:14:32.404 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2020-03-19T21:14:32.405 INFO:teuthology.orchestra.run.smithi170:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2020-03-19T21:14:32.466 INFO:teuthology.orchestra.run.smithi170.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2020-03-19T21:14:32.484 INFO:teuthology.orchestra.run.smithi170.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2020-03-19T21:14:32.527 INFO:teuthology.orchestra.run.smithi170.stderr:sudo: ntpd: command not found 2020-03-19T21:14:32.541 INFO:teuthology.orchestra.run.smithi170.stdout:506 Cannot talk to daemon 2020-03-19T21:14:32.559 INFO:teuthology.orchestra.run.smithi170.stderr:Failed to start ntp.service: Unit ntp.service not found. 2020-03-19T21:14:32.576 INFO:teuthology.orchestra.run.smithi170.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2020-03-19T21:14:32.636 INFO:teuthology.orchestra.run.smithi170.stderr:bash: ntpq: command not found 2020-03-19T21:14:32.640 INFO:teuthology.orchestra.run.smithi170.stdout:210 Number of sources = 4 2020-03-19T21:14:32.640 INFO:teuthology.orchestra.run.smithi170.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2020-03-19T21:14:32.640 INFO:teuthology.orchestra.run.smithi170.stdout:=============================================================================== 2020-03-19T21:14:32.640 INFO:teuthology.orchestra.run.smithi170.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2020-03-19T21:14:32.640 INFO:teuthology.orchestra.run.smithi170.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2020-03-19T21:14:32.640 INFO:teuthology.orchestra.run.smithi170.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2020-03-19T21:14:32.641 INFO:teuthology.orchestra.run.smithi170.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2020-03-19T21:14:32.641 INFO:teuthology.run_tasks:Running task install... 2020-03-19T21:14:32.650 DEBUG:teuthology.task.install:project ceph 2020-03-19T21:14:32.651 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'sha1': '787db24c312ed9d310cd3f194b00548ede9a199c'}} 2020-03-19T21:14:32.651 DEBUG:teuthology.task.install:config {'sha1': '787db24c312ed9d310cd3f194b00548ede9a199c'} 2020-03-19T21:14:32.651 INFO:teuthology.task.install:Using flavor: basic 2020-03-19T21:14:32.658 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2020-03-19T21:14:32.658 INFO:teuthology.task.install:extra packages: [] 2020-03-19T21:14:32.659 INFO:teuthology.orchestra.run.smithi170:> sudo lsb_release -is 2020-03-19T21:14:32.716 INFO:teuthology.orchestra.run.smithi170.stdout:RedHatEnterprise 2020-03-19T21:14:32.717 DEBUG:teuthology.misc:System to be installed: RedHatEnterprise 2020-03-19T21:14:32.718 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'sha1': '787db24c312ed9d310cd3f194b00548ede9a199c', 'exclude_packages': [], 'debuginfo': None, 'install_ceph_packages': True, 'wait_for_package': False, 'downgrade_packages': [], 'project': 'ceph', 'tag': None, 'extra_packages': [], 'branch': None, 'packages': {}, 'repos_only': False, 'extra_system_packages': [], 'flavor': 'basic', 'extras': None} 2020-03-19T21:14:32.718 INFO:teuthology.orchestra.run.smithi170:> sudo lsb_release -is 2020-03-19T21:14:32.793 INFO:teuthology.orchestra.run.smithi170.stdout:RedHatEnterprise 2020-03-19T21:14:32.794 DEBUG:teuthology.misc:System to be installed: RedHatEnterprise 2020-03-19T21:14:32.795 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=787db24c312ed9d310cd3f194b00548ede9a199c 2020-03-19T21:14:33.082 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-sage-testing-2020-03-19-0855/787db24c312ed9d310cd3f194b00548ede9a199c/centos/8/flavors/default/ 2020-03-19T21:14:33.290 INFO:teuthology.task.install.rpm:Package version is 15.1.1-234.g787db24 2020-03-19T21:14:33.464 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-sage-testing-2020-03-19-0855/787db24c312ed9d310cd3f194b00548ede9a199c/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-sage-testing-2020-03-19-0855/787db24c312ed9d310cd3f194b00548ede9a199c/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-sage-testing-2020-03-19-0855/787db24c312ed9d310cd3f194b00548ede9a199c/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2020-03-19T21:14:33.464 INFO:teuthology.orchestra.run.smithi170:> sudo sh -c 'cat > /etc/yum.repos.d/ceph.repo' 2020-03-19T21:14:33.517 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2020-03-19T21:14:33.518 INFO:teuthology.orchestra.run.smithi170:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/787db24c312ed9d310cd3f194b00548ede9a199c/;g' /etc/yum.repos.d/ceph.repo ; fi 2020-03-19T21:14:33.591 INFO:teuthology.orchestra.run.smithi170:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2020-03-19T21:14:33.675 INFO:teuthology.orchestra.run.smithi170:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2020-03-19T21:14:33.743 INFO:teuthology.orchestra.run.smithi170.stdout:check_obsoletes = 1 2020-03-19T21:14:33.744 INFO:teuthology.orchestra.run.smithi170:> sudo yum clean all 2020-03-19T21:14:34.111 INFO:teuthology.orchestra.run.smithi170.stdout:Updating Subscription Management repositories. 2020-03-19T21:14:34.517 INFO:teuthology.orchestra.run.smithi170.stdout:44 files removed 2020-03-19T21:14:34.561 INFO:teuthology.orchestra.run.smithi170:> sudo yum -y install ceph-radosgw 2020-03-19T21:14:34.886 INFO:teuthology.orchestra.run.smithi170.stdout:Updating Subscription Management repositories. 2020-03-19T21:14:35.901 INFO:teuthology.orchestra.run.smithi170.stdout:ceph packages for x86_64 147 kB/s | 69 kB 00:00 2020-03-19T21:14:36.242 INFO:teuthology.orchestra.run.smithi170.stdout:ceph noarch packages 44 kB/s | 14 kB 00:00 2020-03-19T21:14:36.595 INFO:teuthology.orchestra.run.smithi170.stdout:ceph source packages 4.2 kB/s | 1.4 kB 00:00 2020-03-19T21:14:36.707 INFO:teuthology.orchestra.run.smithi170.stdout:Copr repo for ceph-el8 owned by ktdreyer 402 kB/s | 38 kB 00:00 2020-03-19T21:14:36.981 INFO:teuthology.orchestra.run.smithi170.stdout:Extra Packages for Enterprise Linux 24 MB/s | 6.1 MB 00:00 2020-03-19T21:14:38.169 INFO:teuthology.orchestra.run.smithi170.stdout:lab-extras 5.6 MB/s | 30 kB 00:00 2020-03-19T21:14:38.374 INFO:teuthology.orchestra.run.smithi170.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 84 MB/s | 14 MB 00:00 2020-03-19T21:14:40.436 INFO:teuthology.orchestra.run.smithi170.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 79 MB/s | 14 MB 00:00 2020-03-19T21:14:44.002 INFO:teuthology.orchestra.run.smithi170.stdout:Dependencies resolved. 2020-03-19T21:14:44.005 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2020-03-19T21:14:44.005 INFO:teuthology.orchestra.run.smithi170.stdout: Package Arch Version Repository Size 2020-03-19T21:14:44.005 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2020-03-19T21:14:44.005 INFO:teuthology.orchestra.run.smithi170.stdout:Installing: 2020-03-19T21:14:44.005 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-radosgw x86_64 2:15.1.1-234.g787db24.el8 ceph 9.8 M 2020-03-19T21:14:44.006 INFO:teuthology.orchestra.run.smithi170.stdout:Upgrading: 2020-03-19T21:14:44.006 INFO:teuthology.orchestra.run.smithi170.stdout: librados2 x86_64 2:15.1.1-234.g787db24.el8 ceph 3.6 M 2020-03-19T21:14:44.006 INFO:teuthology.orchestra.run.smithi170.stdout: librbd1 x86_64 2:15.1.1-234.g787db24.el8 ceph 1.9 M 2020-03-19T21:14:44.006 INFO:teuthology.orchestra.run.smithi170.stdout:Installing dependencies: 2020-03-19T21:14:44.006 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-base x86_64 2:15.1.1-234.g787db24.el8 ceph 5.5 M 2020-03-19T21:14:44.006 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-common x86_64 2:15.1.1-234.g787db24.el8 ceph 21 M 2020-03-19T21:14:44.006 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-selinux x86_64 2:15.1.1-234.g787db24.el8 ceph 25 k 2020-03-19T21:14:44.007 INFO:teuthology.orchestra.run.smithi170.stdout: libcephfs2 x86_64 2:15.1.1-234.g787db24.el8 ceph 508 k 2020-03-19T21:14:44.007 INFO:teuthology.orchestra.run.smithi170.stdout: libradosstriper1 x86_64 2:15.1.1-234.g787db24.el8 ceph 377 k 2020-03-19T21:14:44.007 INFO:teuthology.orchestra.run.smithi170.stdout: librgw2 x86_64 2:15.1.1-234.g787db24.el8 ceph 6.2 M 2020-03-19T21:14:44.007 INFO:teuthology.orchestra.run.smithi170.stdout: python3-ceph-argparse x86_64 2:15.1.1-234.g787db24.el8 ceph 40 k 2020-03-19T21:14:44.007 INFO:teuthology.orchestra.run.smithi170.stdout: python3-ceph-common x86_64 2:15.1.1-234.g787db24.el8 ceph 60 k 2020-03-19T21:14:44.007 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cephfs x86_64 2:15.1.1-234.g787db24.el8 ceph 162 k 2020-03-19T21:14:44.007 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rados x86_64 2:15.1.1-234.g787db24.el8 ceph 352 k 2020-03-19T21:14:44.007 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rbd x86_64 2:15.1.1-234.g787db24.el8 ceph 309 k 2020-03-19T21:14:44.008 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rgw x86_64 2:15.1.1-234.g787db24.el8 ceph 113 k 2020-03-19T21:14:44.008 INFO:teuthology.orchestra.run.smithi170.stdout: gperftools-libs x86_64 2.7-6.el8 epel 315 k 2020-03-19T21:14:44.008 INFO:teuthology.orchestra.run.smithi170.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2020-03-19T21:14:44.008 INFO:teuthology.orchestra.run.smithi170.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2020-03-19T21:14:44.008 INFO:teuthology.orchestra.run.smithi170.stdout: leveldb x86_64 1.21-2.el8 lab-extras 179 k 2020-03-19T21:14:44.008 INFO:teuthology.orchestra.run.smithi170.stdout: lttng-ust x86_64 2.8.1-11.el8 lab-extras 257 k 2020-03-19T21:14:44.008 INFO:teuthology.orchestra.run.smithi170.stdout: librabbitmq x86_64 0.9.0-1.el8 rhel-8-for-x86_64-baseos-rpms 2020-03-19T21:14:44.009 INFO:teuthology.orchestra.run.smithi170.stdout: 47 k 2020-03-19T21:14:44.009 INFO:teuthology.orchestra.run.smithi170.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 2020-03-19T21:14:44.009 INFO:teuthology.orchestra.run.smithi170.stdout: 44 k 2020-03-19T21:14:44.009 INFO:teuthology.orchestra.run.smithi170.stdout: librdkafka x86_64 0.11.4-1.el8 rhel-8-for-x86_64-appstream-rpms 2020-03-19T21:14:44.009 INFO:teuthology.orchestra.run.smithi170.stdout: 353 k 2020-03-19T21:14:44.009 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:14:44.009 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction Summary 2020-03-19T21:14:44.010 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2020-03-19T21:14:44.010 INFO:teuthology.orchestra.run.smithi170.stdout:Install 21 Packages 2020-03-19T21:14:44.010 INFO:teuthology.orchestra.run.smithi170.stdout:Upgrade 2 Packages 2020-03-19T21:14:44.010 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:14:44.010 INFO:teuthology.orchestra.run.smithi170.stdout:Total download size: 51 M 2020-03-19T21:14:44.011 INFO:teuthology.orchestra.run.smithi170.stdout:Downloading Packages: 2020-03-19T21:14:45.744 INFO:teuthology.orchestra.run.smithi170.stdout:(1/23): ceph-base-15.1.1-234.g787db24.el8.x86_6 3.2 MB/s | 5.5 MB 00:01 2020-03-19T21:14:45.778 INFO:teuthology.orchestra.run.smithi170.stdout:(2/23): ceph-selinux-15.1.1-234.g787db24.el8.x8 728 kB/s | 25 kB 00:00 2020-03-19T21:14:45.847 INFO:teuthology.orchestra.run.smithi170.stdout:(3/23): libcephfs2-15.1.1-234.g787db24.el8.x86_ 7.1 MB/s | 508 kB 00:00 2020-03-19T21:14:45.914 INFO:teuthology.orchestra.run.smithi170.stdout:(4/23): libradosstriper1-15.1.1-234.g787db24.el 5.5 MB/s | 377 kB 00:00 2020-03-19T21:14:46.495 INFO:teuthology.orchestra.run.smithi170.stdout:(5/23): librgw2-15.1.1-234.g787db24.el8.x86_64. 11 MB/s | 6.2 MB 00:00 2020-03-19T21:14:48.321 INFO:teuthology.orchestra.run.smithi170.stdout:(6/23): python3-ceph-argparse-15.1.1-234.g787db 22 kB/s | 40 kB 00:01 2020-03-19T21:14:48.386 INFO:teuthology.orchestra.run.smithi170.stdout:(7/23): python3-ceph-common-15.1.1-234.g787db24 917 kB/s | 60 kB 00:00 2020-03-19T21:14:48.473 INFO:teuthology.orchestra.run.smithi170.stdout:(8/23): python3-cephfs-15.1.1-234.g787db24.el8. 1.8 MB/s | 162 kB 00:00 2020-03-19T21:14:48.602 INFO:teuthology.orchestra.run.smithi170.stdout:(9/23): python3-rados-15.1.1-234.g787db24.el8.x 2.7 MB/s | 352 kB 00:00 2020-03-19T21:14:48.740 INFO:teuthology.orchestra.run.smithi170.stdout:(10/23): python3-rbd-15.1.1-234.g787db24.el8.x8 2.2 MB/s | 309 kB 00:00 2020-03-19T21:14:48.921 INFO:teuthology.orchestra.run.smithi170.stdout:(11/23): ceph-common-15.1.1-234.g787db24.el8.x8 4.2 MB/s | 21 MB 00:04 2020-03-19T21:14:48.923 INFO:teuthology.orchestra.run.smithi170.stdout:(12/23): python3-rgw-15.1.1-234.g787db24.el8.x8 620 kB/s | 113 kB 00:00 2020-03-19T21:14:48.985 INFO:teuthology.orchestra.run.smithi170.stdout:(13/23): liboath-2.6.2-3.el8.x86_64.rpm 948 kB/s | 59 kB 00:00 2020-03-19T21:14:49.016 INFO:teuthology.orchestra.run.smithi170.stdout:(14/23): gperftools-libs-2.7-6.el8.x86_64.rpm 3.3 MB/s | 315 kB 00:00 2020-03-19T21:14:49.016 INFO:teuthology.orchestra.run.smithi170.stdout:(15/23): libunwind-1.3.1-3.el8.x86_64.rpm 2.4 MB/s | 75 kB 00:00 2020-03-19T21:14:49.021 INFO:teuthology.orchestra.run.smithi170.stdout:(16/23): leveldb-1.21-2.el8.x86_64.rpm 33 MB/s | 179 kB 00:00 2020-03-19T21:14:49.022 INFO:teuthology.orchestra.run.smithi170.stdout:(17/23): lttng-ust-2.8.1-11.el8.x86_64.rpm 44 MB/s | 257 kB 00:00 2020-03-19T21:14:49.062 INFO:teuthology.orchestra.run.smithi170.stdout:(18/23): librabbitmq-0.9.0-1.el8.x86_64.rpm 1.1 MB/s | 47 kB 00:00 2020-03-19T21:14:49.069 INFO:teuthology.orchestra.run.smithi170.stdout:(19/23): python3-prettytable-0.7.2-14.el8.noarc 942 kB/s | 44 kB 00:00 2020-03-19T21:14:49.088 INFO:teuthology.orchestra.run.smithi170.stdout:(20/23): librdkafka-0.11.4-1.el8.x86_64.rpm 13 MB/s | 353 kB 00:00 2020-03-19T21:14:56.341 INFO:teuthology.orchestra.run.smithi170.stdout:(21/23): librbd1-15.1.1-234.g787db24.el8.x86_64 270 kB/s | 1.9 MB 00:07 2020-03-19T21:14:56.618 INFO:teuthology.orchestra.run.smithi170.stdout:(22/23): librados2-15.1.1-234.g787db24.el8.x86_ 485 kB/s | 3.6 MB 00:07 2020-03-19T21:14:58.242 INFO:teuthology.orchestra.run.smithi170.stdout:(23/23): ceph-radosgw-15.1.1-234.g787db24.el8.x 705 kB/s | 9.8 MB 00:14 2020-03-19T21:14:58.244 INFO:teuthology.orchestra.run.smithi170.stdout:-------------------------------------------------------------------------------- 2020-03-19T21:14:58.245 INFO:teuthology.orchestra.run.smithi170.stdout:Total 3.6 MB/s | 51 MB 00:14 2020-03-19T21:14:58.252 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction check 2020-03-19T21:14:58.297 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction check succeeded. 2020-03-19T21:14:58.298 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction test 2020-03-19T21:14:58.654 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction test succeeded. 2020-03-19T21:14:58.654 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction 2020-03-19T21:14:59.722 INFO:teuthology.orchestra.run.smithi170.stdout: Preparing : 1/1 2020-03-19T21:14:59.868 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 1/25 2020-03-19T21:15:00.287 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 1/25 2020-03-19T21:15:00.443 INFO:teuthology.orchestra.run.smithi170.stdout: Upgrading : librados2-2:15.1.1-234.g787db24.el8.x86_64 2/25 2020-03-19T21:15:00.568 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: librados2-2:15.1.1-234.g787db24.el8.x86_64 2/25 2020-03-19T21:15:00.740 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-rados-2:15.1.1-234.g787db24.el8.x86_64 3/25 2020-03-19T21:15:00.985 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : liboath-2.6.2-3.el8.x86_64 4/25 2020-03-19T21:15:01.210 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libcephfs2-2:15.1.1-234.g787db24.el8.x86_64 5/25 2020-03-19T21:15:01.447 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: libcephfs2-2:15.1.1-234.g787db24.el8.x86_64 5/25 2020-03-19T21:15:01.643 INFO:teuthology.orchestra.run.smithi170.stdout: Upgrading : librbd1-2:15.1.1-234.g787db24.el8.x86_64 6/25 2020-03-19T21:15:01.738 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: librbd1-2:15.1.1-234.g787db24.el8.x86_64 6/25 2020-03-19T21:15:01.859 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : librdkafka-0.11.4-1.el8.x86_64 7/25 2020-03-19T21:15:01.911 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: librdkafka-0.11.4-1.el8.x86_64 7/25 2020-03-19T21:15:02.620 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : librabbitmq-0.9.0-1.el8.x86_64 8/25 2020-03-19T21:15:02.742 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : librgw2-2:15.1.1-234.g787db24.el8.x86_64 9/25 2020-03-19T21:15:02.842 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: librgw2-2:15.1.1-234.g787db24.el8.x86_64 9/25 2020-03-19T21:15:02.943 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-ceph-argparse-2:15.1.1-234.g787db24.el8.x8 10/25 2020-03-19T21:15:03.137 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-cephfs-2:15.1.1-234.g787db24.el8.x86_64 11/25 2020-03-19T21:15:03.313 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-rgw-2:15.1.1-234.g787db24.el8.x86_64 12/25 2020-03-19T21:15:03.509 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-rbd-2:15.1.1-234.g787db24.el8.x86_64 13/25 2020-03-19T21:15:03.634 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libradosstriper1-2:15.1.1-234.g787db24.el8.x86_64 14/25 2020-03-19T21:15:03.711 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: libradosstriper1-2:15.1.1-234.g787db24.el8.x86_64 14/25 2020-03-19T21:15:03.877 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 15/25 2020-03-19T21:15:04.106 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 16/25 2020-03-19T21:15:04.334 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : gperftools-libs-2.7-6.el8.x86_64 17/25 2020-03-19T21:15:04.494 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : leveldb-1.21-2.el8.x86_64 18/25 2020-03-19T21:15:04.667 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-ceph-common-2:15.1.1-234.g787db24.el8.x86_ 19/25 2020-03-19T21:15:08.284 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-common-2:15.1.1-234.g787db24.el8.x86_64 20/25 2020-03-19T21:15:08.475 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-common-2:15.1.1-234.g787db24.el8.x86_64 20/25 2020-03-19T21:15:09.157 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-common-2:15.1.1-234.g787db24.el8.x86_64 20/25 2020-03-19T21:15:09.409 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-base-2:15.1.1-234.g787db24.el8.x86_64 21/25 2020-03-19T21:15:09.498 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-base-2:15.1.1-234.g787db24.el8.x86_64 21/25 2020-03-19T21:15:09.616 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-selinux-2:15.1.1-234.g787db24.el8.x86_64 22/25 2020-03-19T21:15:25.848 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-selinux-2:15.1.1-234.g787db24.el8.x86_64 22/25 2020-03-19T21:15:25.848 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /sys 2020-03-19T21:15:25.848 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /proc 2020-03-19T21:15:25.848 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /dev 2020-03-19T21:15:25.848 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /run 2020-03-19T21:15:25.849 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /mnt 2020-03-19T21:15:25.849 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /var/tmp 2020-03-19T21:15:25.849 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /home 2020-03-19T21:15:25.849 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /tmp 2020-03-19T21:15:25.849 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /dev 2020-03-19T21:15:25.849 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:15:26.922 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-radosgw-2:15.1.1-234.g787db24.el8.x86_64 23/25 2020-03-19T21:15:26.990 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-radosgw-2:15.1.1-234.g787db24.el8.x86_64 23/25 2020-03-19T21:15:26.990 INFO:teuthology.orchestra.run.smithi170.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 24/25 2020-03-19T21:15:27.199 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 24/25 2020-03-19T21:15:27.199 INFO:teuthology.orchestra.run.smithi170.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 25/25 2020-03-19T21:15:27.875 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 25/25 2020-03-19T21:15:27.875 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-base-2:15.1.1-234.g787db24.el8.x86_64 1/25 2020-03-19T21:15:27.875 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-common-2:15.1.1-234.g787db24.el8.x86_64 2/25 2020-03-19T21:15:27.875 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-radosgw-2:15.1.1-234.g787db24.el8.x86_64 3/25 2020-03-19T21:15:27.875 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-selinux-2:15.1.1-234.g787db24.el8.x86_64 4/25 2020-03-19T21:15:27.875 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libcephfs2-2:15.1.1-234.g787db24.el8.x86_64 5/25 2020-03-19T21:15:27.876 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libradosstriper1-2:15.1.1-234.g787db24.el8.x86_64 6/25 2020-03-19T21:15:27.876 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librgw2-2:15.1.1-234.g787db24.el8.x86_64 7/25 2020-03-19T21:15:27.876 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-ceph-argparse-2:15.1.1-234.g787db24.el8.x8 8/25 2020-03-19T21:15:27.876 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-ceph-common-2:15.1.1-234.g787db24.el8.x86_ 9/25 2020-03-19T21:15:27.876 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-cephfs-2:15.1.1-234.g787db24.el8.x86_64 10/25 2020-03-19T21:15:27.876 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-rados-2:15.1.1-234.g787db24.el8.x86_64 11/25 2020-03-19T21:15:27.876 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-rbd-2:15.1.1-234.g787db24.el8.x86_64 12/25 2020-03-19T21:15:27.877 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-rgw-2:15.1.1-234.g787db24.el8.x86_64 13/25 2020-03-19T21:15:27.877 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : gperftools-libs-2.7-6.el8.x86_64 14/25 2020-03-19T21:15:27.877 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 15/25 2020-03-19T21:15:27.877 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 16/25 2020-03-19T21:15:27.877 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : leveldb-1.21-2.el8.x86_64 17/25 2020-03-19T21:15:27.877 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 18/25 2020-03-19T21:15:27.877 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librabbitmq-0.9.0-1.el8.x86_64 19/25 2020-03-19T21:15:27.877 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 20/25 2020-03-19T21:15:27.878 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librdkafka-0.11.4-1.el8.x86_64 21/25 2020-03-19T21:15:27.878 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librados2-2:15.1.1-234.g787db24.el8.x86_64 22/25 2020-03-19T21:15:27.878 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 23/25 2020-03-19T21:15:27.878 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librbd1-2:15.1.1-234.g787db24.el8.x86_64 24/25 2020-03-19T21:15:30.763 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 25/25 2020-03-19T21:15:30.764 INFO:teuthology.orchestra.run.smithi170.stdout:Installed products updated. 2020-03-19T21:15:30.942 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:15:30.942 INFO:teuthology.orchestra.run.smithi170.stdout:Upgraded: 2020-03-19T21:15:30.942 INFO:teuthology.orchestra.run.smithi170.stdout: librados2-2:15.1.1-234.g787db24.el8.x86_64 2020-03-19T21:15:30.942 INFO:teuthology.orchestra.run.smithi170.stdout: librbd1-2:15.1.1-234.g787db24.el8.x86_64 2020-03-19T21:15:30.943 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:15:30.943 INFO:teuthology.orchestra.run.smithi170.stdout:Installed: 2020-03-19T21:15:30.943 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-radosgw-2:15.1.1-234.g787db24.el8.x86_64 2020-03-19T21:15:30.943 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-base-2:15.1.1-234.g787db24.el8.x86_64 2020-03-19T21:15:30.944 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-common-2:15.1.1-234.g787db24.el8.x86_64 2020-03-19T21:15:30.944 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-selinux-2:15.1.1-234.g787db24.el8.x86_64 2020-03-19T21:15:30.944 INFO:teuthology.orchestra.run.smithi170.stdout: libcephfs2-2:15.1.1-234.g787db24.el8.x86_64 2020-03-19T21:15:30.944 INFO:teuthology.orchestra.run.smithi170.stdout: libradosstriper1-2:15.1.1-234.g787db24.el8.x86_64 2020-03-19T21:15:30.945 INFO:teuthology.orchestra.run.smithi170.stdout: librgw2-2:15.1.1-234.g787db24.el8.x86_64 2020-03-19T21:15:30.945 INFO:teuthology.orchestra.run.smithi170.stdout: python3-ceph-argparse-2:15.1.1-234.g787db24.el8.x86_64 2020-03-19T21:15:30.945 INFO:teuthology.orchestra.run.smithi170.stdout: python3-ceph-common-2:15.1.1-234.g787db24.el8.x86_64 2020-03-19T21:15:30.945 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cephfs-2:15.1.1-234.g787db24.el8.x86_64 2020-03-19T21:15:30.946 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rados-2:15.1.1-234.g787db24.el8.x86_64 2020-03-19T21:15:30.946 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rbd-2:15.1.1-234.g787db24.el8.x86_64 2020-03-19T21:15:30.946 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rgw-2:15.1.1-234.g787db24.el8.x86_64 2020-03-19T21:15:30.946 INFO:teuthology.orchestra.run.smithi170.stdout: gperftools-libs-2.7-6.el8.x86_64 2020-03-19T21:15:30.947 INFO:teuthology.orchestra.run.smithi170.stdout: liboath-2.6.2-3.el8.x86_64 2020-03-19T21:15:30.947 INFO:teuthology.orchestra.run.smithi170.stdout: libunwind-1.3.1-3.el8.x86_64 2020-03-19T21:15:30.947 INFO:teuthology.orchestra.run.smithi170.stdout: leveldb-1.21-2.el8.x86_64 2020-03-19T21:15:30.947 INFO:teuthology.orchestra.run.smithi170.stdout: lttng-ust-2.8.1-11.el8.x86_64 2020-03-19T21:15:30.948 INFO:teuthology.orchestra.run.smithi170.stdout: librabbitmq-0.9.0-1.el8.x86_64 2020-03-19T21:15:30.948 INFO:teuthology.orchestra.run.smithi170.stdout: python3-prettytable-0.7.2-14.el8.noarch 2020-03-19T21:15:30.948 INFO:teuthology.orchestra.run.smithi170.stdout: librdkafka-0.11.4-1.el8.x86_64 2020-03-19T21:15:30.948 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:15:30.949 INFO:teuthology.orchestra.run.smithi170.stdout:Complete! 2020-03-19T21:15:31.078 INFO:teuthology.orchestra.run.smithi170:> sudo yum -y install ceph-test 2020-03-19T21:15:31.419 INFO:teuthology.orchestra.run.smithi170.stdout:Updating Subscription Management repositories. 2020-03-19T21:15:32.560 INFO:teuthology.orchestra.run.smithi170.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 76 kB/s | 2.4 kB 00:00 2020-03-19T21:15:32.642 INFO:teuthology.orchestra.run.smithi170.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 71 kB/s | 2.8 kB 00:00 2020-03-19T21:15:33.411 INFO:teuthology.orchestra.run.smithi170.stdout:Dependencies resolved. 2020-03-19T21:15:33.412 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2020-03-19T21:15:33.412 INFO:teuthology.orchestra.run.smithi170.stdout: Package Arch Version Repository Size 2020-03-19T21:15:33.412 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2020-03-19T21:15:33.412 INFO:teuthology.orchestra.run.smithi170.stdout:Installing: 2020-03-19T21:15:33.412 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-test x86_64 2:15.1.1-234.g787db24.el8 2020-03-19T21:15:33.413 INFO:teuthology.orchestra.run.smithi170.stdout: ceph 42 M 2020-03-19T21:15:33.413 INFO:teuthology.orchestra.run.smithi170.stdout:Installing dependencies: 2020-03-19T21:15:33.413 INFO:teuthology.orchestra.run.smithi170.stdout: xmlstarlet x86_64 1.6.1-11.el8 epel 69 k 2020-03-19T21:15:33.413 INFO:teuthology.orchestra.run.smithi170.stdout: libxslt x86_64 1.1.32-3.el8 rhel-8-for-x86_64-baseos-rpms 249 k 2020-03-19T21:15:33.413 INFO:teuthology.orchestra.run.smithi170.stdout: jq x86_64 1.5-12.el8 rhel-8-for-x86_64-appstream-rpms 161 k 2020-03-19T21:15:33.413 INFO:teuthology.orchestra.run.smithi170.stdout: socat x86_64 1.7.3.2-6.el8 rhel-8-for-x86_64-appstream-rpms 298 k 2020-03-19T21:15:33.413 INFO:teuthology.orchestra.run.smithi170.stdout: oniguruma x86_64 6.8.2-1.el8 rhel-8-for-x86_64-appstream-rpms 188 k 2020-03-19T21:15:33.414 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:15:33.414 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction Summary 2020-03-19T21:15:33.414 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2020-03-19T21:15:33.414 INFO:teuthology.orchestra.run.smithi170.stdout:Install 6 Packages 2020-03-19T21:15:33.414 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:15:33.415 INFO:teuthology.orchestra.run.smithi170.stdout:Total download size: 43 M 2020-03-19T21:15:33.415 INFO:teuthology.orchestra.run.smithi170.stdout:Installed size: 185 M 2020-03-19T21:15:33.415 INFO:teuthology.orchestra.run.smithi170.stdout:Downloading Packages: 2020-03-19T21:15:33.516 INFO:teuthology.orchestra.run.smithi170.stdout:(1/6): libxslt-1.1.32-3.el8.x86_64.rpm 2.4 MB/s | 249 kB 00:00 2020-03-19T21:15:33.586 INFO:teuthology.orchestra.run.smithi170.stdout:(2/6): xmlstarlet-1.6.1-11.el8.x86_64.rpm 401 kB/s | 69 kB 00:00 2020-03-19T21:15:33.612 INFO:teuthology.orchestra.run.smithi170.stdout:(3/6): jq-1.5-12.el8.x86_64.rpm 1.7 MB/s | 161 kB 00:00 2020-03-19T21:15:33.630 INFO:teuthology.orchestra.run.smithi170.stdout:(4/6): socat-1.7.3.2-6.el8.x86_64.rpm 6.7 MB/s | 298 kB 00:00 2020-03-19T21:15:33.638 INFO:teuthology.orchestra.run.smithi170.stdout:(5/6): oniguruma-6.8.2-1.el8.x86_64.rpm 7.1 MB/s | 188 kB 00:00 2020-03-19T21:15:36.948 INFO:teuthology.orchestra.run.smithi170.stdout:(6/6): ceph-test-15.1.1-234.g787db24.el8.x86_64 12 MB/s | 42 MB 00:03 2020-03-19T21:15:36.950 INFO:teuthology.orchestra.run.smithi170.stdout:-------------------------------------------------------------------------------- 2020-03-19T21:15:36.950 INFO:teuthology.orchestra.run.smithi170.stdout:Total 12 MB/s | 43 MB 00:03 2020-03-19T21:15:36.957 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction check 2020-03-19T21:15:36.979 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction check succeeded. 2020-03-19T21:15:36.979 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction test 2020-03-19T21:15:37.253 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction test succeeded. 2020-03-19T21:15:37.253 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction 2020-03-19T21:15:37.721 INFO:teuthology.orchestra.run.smithi170.stdout: Preparing : 1/1 2020-03-19T21:15:37.878 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : oniguruma-6.8.2-1.el8.x86_64 1/6 2020-03-19T21:15:37.986 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: oniguruma-6.8.2-1.el8.x86_64 1/6 2020-03-19T21:15:38.175 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : jq-1.5-12.el8.x86_64 2/6 2020-03-19T21:15:38.386 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : socat-1.7.3.2-6.el8.x86_64 3/6 2020-03-19T21:15:38.580 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libxslt-1.1.32-3.el8.x86_64 4/6 2020-03-19T21:15:42.562 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : xmlstarlet-1.6.1-11.el8.x86_64 5/6 2020-03-19T21:15:42.825 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-test-2:15.1.1-234.g787db24.el8.x86_64 6/6 2020-03-19T21:15:43.061 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-test-2:15.1.1-234.g787db24.el8.x86_64 6/6 2020-03-19T21:15:43.062 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-test-2:15.1.1-234.g787db24.el8.x86_64 1/6 2020-03-19T21:15:43.062 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : xmlstarlet-1.6.1-11.el8.x86_64 2/6 2020-03-19T21:15:43.062 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libxslt-1.1.32-3.el8.x86_64 3/6 2020-03-19T21:15:43.062 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : jq-1.5-12.el8.x86_64 4/6 2020-03-19T21:15:43.062 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : socat-1.7.3.2-6.el8.x86_64 5/6 2020-03-19T21:15:45.228 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : oniguruma-6.8.2-1.el8.x86_64 6/6 2020-03-19T21:15:45.228 INFO:teuthology.orchestra.run.smithi170.stdout:Installed products updated. 2020-03-19T21:15:45.397 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:15:45.398 INFO:teuthology.orchestra.run.smithi170.stdout:Installed: 2020-03-19T21:15:45.398 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-test-2:15.1.1-234.g787db24.el8.x86_64 xmlstarlet-1.6.1-11.el8.x86_64 2020-03-19T21:15:45.398 INFO:teuthology.orchestra.run.smithi170.stdout: libxslt-1.1.32-3.el8.x86_64 jq-1.5-12.el8.x86_64 2020-03-19T21:15:45.398 INFO:teuthology.orchestra.run.smithi170.stdout: socat-1.7.3.2-6.el8.x86_64 oniguruma-6.8.2-1.el8.x86_64 2020-03-19T21:15:45.398 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:15:45.398 INFO:teuthology.orchestra.run.smithi170.stdout:Complete! 2020-03-19T21:15:45.586 INFO:teuthology.orchestra.run.smithi170:> sudo yum -y install ceph 2020-03-19T21:15:45.915 INFO:teuthology.orchestra.run.smithi170.stdout:Updating Subscription Management repositories. 2020-03-19T21:15:46.479 INFO:teuthology.orchestra.run.smithi170.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 73 kB/s | 2.4 kB 00:00 2020-03-19T21:15:46.551 INFO:teuthology.orchestra.run.smithi170.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 93 kB/s | 2.8 kB 00:00 2020-03-19T21:15:47.314 INFO:teuthology.orchestra.run.smithi170.stdout:Dependencies resolved. 2020-03-19T21:15:47.317 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2020-03-19T21:15:47.318 INFO:teuthology.orchestra.run.smithi170.stdout: Package ArchVersion Repository Size 2020-03-19T21:15:47.318 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2020-03-19T21:15:47.318 INFO:teuthology.orchestra.run.smithi170.stdout:Installing: 2020-03-19T21:15:47.318 INFO:teuthology.orchestra.run.smithi170.stdout: ceph x86_642:15.1.1-234.g787db24.el8 ceph 6.4 k 2020-03-19T21:15:47.318 INFO:teuthology.orchestra.run.smithi170.stdout:Installing dependencies: 2020-03-19T21:15:47.318 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mds x86_642:15.1.1-234.g787db24.el8 ceph 2.0 M 2020-03-19T21:15:47.318 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr x86_642:15.1.1-234.g787db24.el8 ceph 1.4 M 2020-03-19T21:15:47.319 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mon x86_642:15.1.1-234.g787db24.el8 ceph 4.4 M 2020-03-19T21:15:47.319 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-osd x86_642:15.1.1-234.g787db24.el8 ceph 17 M 2020-03-19T21:15:47.319 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-modules-core noarch2:15.1.1-234.g787db24.el8 ceph-noarch 163 k 2020-03-19T21:15:47.319 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cheroot noarch8.2.1-1.el8 copr:copr.fedorainfracloud.org:ktdreyer:ceph-el8 145 k 2020-03-19T21:15:47.319 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cherrypy noarch18.4.0-1.el8 copr:copr.fedorainfracloud.org:ktdreyer:ceph-el8 385 k 2020-03-19T21:15:47.319 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco noarch6.2-6.el8 copr:copr.fedorainfracloud.org:ktdreyer:ceph-el8 11 k 2020-03-19T21:15:47.319 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco-functools noarch2.0-4.el8 copr:copr.fedorainfracloud.org:ktdreyer:ceph-el8 18 k 2020-03-19T21:15:47.319 INFO:teuthology.orchestra.run.smithi170.stdout: python3-logutils noarch0.3.5-11.el8 copr:copr.fedorainfracloud.org:ktdreyer:ceph-el8 49 k 2020-03-19T21:15:47.320 INFO:teuthology.orchestra.run.smithi170.stdout: python3-more-itertools noarch7.2.0-3.el8 copr:copr.fedorainfracloud.org:ktdreyer:ceph-el8 59 k 2020-03-19T21:15:47.320 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pecan noarch1.3.2-9.el8 copr:copr.fedorainfracloud.org:ktdreyer:ceph-el8 283 k 2020-03-19T21:15:47.320 INFO:teuthology.orchestra.run.smithi170.stdout: python3-portend noarch2.6-1.el8 copr:copr.fedorainfracloud.org:ktdreyer:ceph-el8 16 k 2020-03-19T21:15:47.320 INFO:teuthology.orchestra.run.smithi170.stdout: python3-simplegeneric noarch0.8.1-17.el8 copr:copr.fedorainfracloud.org:ktdreyer:ceph-el8 19 k 2020-03-19T21:15:47.320 INFO:teuthology.orchestra.run.smithi170.stdout: python3-singledispatch noarch3.4.0.3-18.el8 copr:copr.fedorainfracloud.org:ktdreyer:ceph-el8 24 k 2020-03-19T21:15:47.320 INFO:teuthology.orchestra.run.smithi170.stdout: python3-tempora noarch1.14.1-5.el8 copr:copr.fedorainfracloud.org:ktdreyer:ceph-el8 30 k 2020-03-19T21:15:47.320 INFO:teuthology.orchestra.run.smithi170.stdout: python3-trustme noarch0.5.2-4.el8 copr:copr.fedorainfracloud.org:ktdreyer:ceph-el8 27 k 2020-03-19T21:15:47.321 INFO:teuthology.orchestra.run.smithi170.stdout: python3-zc-lockfile noarch2.0-2.el8 copr:copr.fedorainfracloud.org:ktdreyer:ceph-el8 23 k 2020-03-19T21:15:47.321 INFO:teuthology.orchestra.run.smithi170.stdout: python3-bcrypt x86_643.1.6-2.el8.1 epel 44 k 2020-03-19T21:15:47.321 INFO:teuthology.orchestra.run.smithi170.stdout: python3-beautifulsoup4 noarch4.6.3-2.el8.1 epel 185 k 2020-03-19T21:15:47.321 INFO:teuthology.orchestra.run.smithi170.stdout: python3-waitress noarch1.2.1-2.el8.1 epel 241 k 2020-03-19T21:15:47.321 INFO:teuthology.orchestra.run.smithi170.stdout: python3-webob noarch1.8.5-1.el8.1 epel 251 k 2020-03-19T21:15:47.321 INFO:teuthology.orchestra.run.smithi170.stdout: python3-webtest noarch2.0.33-1.el8 epel 85 k 2020-03-19T21:15:47.321 INFO:teuthology.orchestra.run.smithi170.stdout: python3-webencodings noarch0.5.1-6.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2020-03-19T21:15:47.322 INFO:teuthology.orchestra.run.smithi170.stdout: python3-lxml x86_644.2.3-1.el8 rhel-8-for-x86_64-appstream-rpms 1.5 M 2020-03-19T21:15:47.322 INFO:teuthology.orchestra.run.smithi170.stdout: python3-werkzeug noarch0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2020-03-19T21:15:47.322 INFO:teuthology.orchestra.run.smithi170.stdout: python3-mako noarch1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2020-03-19T21:15:47.322 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pytz noarch2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2020-03-19T21:15:47.322 INFO:teuthology.orchestra.run.smithi170.stdout: python3-markupsafe x86_640.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2020-03-19T21:15:47.322 INFO:teuthology.orchestra.run.smithi170.stdout:Installing weak dependencies: 2020-03-19T21:15:47.322 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cssselect noarch0.9.2-10.el8 epel 40 k 2020-03-19T21:15:47.323 INFO:teuthology.orchestra.run.smithi170.stdout: python3-html5lib noarch1:0.999999999-6.el8 rhel-8-for-x86_64-appstream-rpms 214 k 2020-03-19T21:15:47.323 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:15:47.323 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction Summary 2020-03-19T21:15:47.323 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2020-03-19T21:15:47.323 INFO:teuthology.orchestra.run.smithi170.stdout:Install 32 Packages 2020-03-19T21:15:47.323 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:15:47.324 INFO:teuthology.orchestra.run.smithi170.stdout:Total download size: 29 M 2020-03-19T21:15:47.324 INFO:teuthology.orchestra.run.smithi170.stdout:Installed size: 103 M 2020-03-19T21:15:47.324 INFO:teuthology.orchestra.run.smithi170.stdout:Downloading Packages: 2020-03-19T21:15:47.459 INFO:teuthology.orchestra.run.smithi170.stdout:(1/32): ceph-15.1.1-234.g787db24.el8.x86_64.rpm 47 kB/s | 6.4 kB 00:00 2020-03-19T21:15:47.751 INFO:teuthology.orchestra.run.smithi170.stdout:(2/32): ceph-mds-15.1.1-234.g787db24.el8.x86_64 4.7 MB/s | 2.0 MB 00:00 2020-03-19T21:15:47.755 INFO:teuthology.orchestra.run.smithi170.stdout:(3/32): ceph-mgr-15.1.1-234.g787db24.el8.x86_64 3.3 MB/s | 1.4 MB 00:00 2020-03-19T21:15:47.827 INFO:teuthology.orchestra.run.smithi170.stdout:(4/32): ceph-mgr-modules-core-15.1.1-234.g787db 2.2 MB/s | 163 kB 00:00 2020-03-19T21:15:47.886 INFO:teuthology.orchestra.run.smithi170.stdout:(5/32): python3-cheroot-8.2.1-1.el8.noarch.rpm 2.4 MB/s | 145 kB 00:00 2020-03-19T21:15:47.908 INFO:teuthology.orchestra.run.smithi170.stdout:(6/32): python3-cherrypy-18.4.0-1.el8.noarch.rp 18 MB/s | 385 kB 00:00 2020-03-19T21:15:47.924 INFO:teuthology.orchestra.run.smithi170.stdout:(7/32): ceph-mon-15.1.1-234.g787db24.el8.x86_64 9.4 MB/s | 4.4 MB 00:00 2020-03-19T21:15:47.924 INFO:teuthology.orchestra.run.smithi170.stdout:(8/32): python3-jaraco-6.2-6.el8.noarch.rpm 704 kB/s | 11 kB 00:00 2020-03-19T21:15:47.937 INFO:teuthology.orchestra.run.smithi170.stdout:(9/32): python3-logutils-0.3.5-11.el8.noarch.rp 3.9 MB/s | 49 kB 00:00 2020-03-19T21:15:47.945 INFO:teuthology.orchestra.run.smithi170.stdout:(10/32): python3-more-itertools-7.2.0-3.el8.noa 6.8 MB/s | 59 kB 00:00 2020-03-19T21:15:47.957 INFO:teuthology.orchestra.run.smithi170.stdout:(11/32): python3-pecan-1.3.2-9.el8.noarch.rpm 23 MB/s | 283 kB 00:00 2020-03-19T21:15:47.966 INFO:teuthology.orchestra.run.smithi170.stdout:(12/32): python3-portend-2.6-1.el8.noarch.rpm 1.9 MB/s | 16 kB 00:00 2020-03-19T21:15:47.967 INFO:teuthology.orchestra.run.smithi170.stdout:(13/32): python3-jaraco-functools-2.0-4.el8.noa 418 kB/s | 18 kB 00:00 2020-03-19T21:15:47.974 INFO:teuthology.orchestra.run.smithi170.stdout:(14/32): python3-simplegeneric-0.8.1-17.el8.noa 2.3 MB/s | 19 kB 00:00 2020-03-19T21:15:47.978 INFO:teuthology.orchestra.run.smithi170.stdout:(15/32): python3-singledispatch-3.4.0.3-18.el8. 2.1 MB/s | 24 kB 00:00 2020-03-19T21:15:47.988 INFO:teuthology.orchestra.run.smithi170.stdout:(16/32): python3-trustme-0.5.2-4.el8.noarch.rpm 2.6 MB/s | 27 kB 00:00 2020-03-19T21:15:47.998 INFO:teuthology.orchestra.run.smithi170.stdout:(17/32): python3-zc-lockfile-2.0-2.el8.noarch.r 2.3 MB/s | 23 kB 00:00 2020-03-19T21:15:48.006 INFO:teuthology.orchestra.run.smithi170.stdout:(18/32): python3-tempora-1.14.1-5.el8.noarch.rp 942 kB/s | 30 kB 00:00 2020-03-19T21:15:48.074 INFO:teuthology.orchestra.run.smithi170.stdout:(19/32): python3-bcrypt-3.1.6-2.el8.1.x86_64.rp 578 kB/s | 44 kB 00:00 2020-03-19T21:15:48.084 INFO:teuthology.orchestra.run.smithi170.stdout:(20/32): python3-beautifulsoup4-4.6.3-2.el8.1.n 2.3 MB/s | 185 kB 00:00 2020-03-19T21:15:48.093 INFO:teuthology.orchestra.run.smithi170.stdout:(21/32): python3-cssselect-0.9.2-10.el8.noarch. 2.1 MB/s | 40 kB 00:00 2020-03-19T21:15:48.115 INFO:teuthology.orchestra.run.smithi170.stdout:(22/32): python3-waitress-1.2.1-2.el8.1.noarch. 7.5 MB/s | 241 kB 00:00 2020-03-19T21:15:48.131 INFO:teuthology.orchestra.run.smithi170.stdout:(23/32): python3-webtest-2.0.33-1.el8.noarch.rp 5.2 MB/s | 85 kB 00:00 2020-03-19T21:15:48.149 INFO:teuthology.orchestra.run.smithi170.stdout:(24/32): python3-webob-1.8.5-1.el8.1.noarch.rpm 4.4 MB/s | 251 kB 00:00 2020-03-19T21:15:48.162 INFO:teuthology.orchestra.run.smithi170.stdout:(25/32): python3-webencodings-0.5.1-6.el8.noarc 873 kB/s | 27 kB 00:00 2020-03-19T21:15:48.193 INFO:teuthology.orchestra.run.smithi170.stdout:(26/32): python3-werkzeug-0.12.2-4.el8.noarch.r 14 MB/s | 457 kB 00:00 2020-03-19T21:15:48.199 INFO:teuthology.orchestra.run.smithi170.stdout:(27/32): python3-lxml-4.2.3-1.el8.x86_64.rpm 30 MB/s | 1.5 MB 00:00 2020-03-19T21:15:48.219 INFO:teuthology.orchestra.run.smithi170.stdout:(28/32): python3-mako-1.0.6-13.el8.noarch.rpm 6.1 MB/s | 157 kB 00:00 2020-03-19T21:15:48.232 INFO:teuthology.orchestra.run.smithi170.stdout:(29/32): python3-html5lib-0.999999999-6.el8.noa 7.4 MB/s | 214 kB 00:00 2020-03-19T21:15:48.245 INFO:teuthology.orchestra.run.smithi170.stdout:(30/32): python3-pytz-2017.2-9.el8.noarch.rpm 2.0 MB/s | 54 kB 00:00 2020-03-19T21:15:48.259 INFO:teuthology.orchestra.run.smithi170.stdout:(31/32): python3-markupsafe-0.23-19.el8.x86_64. 1.4 MB/s | 39 kB 00:00 2020-03-19T21:15:48.564 INFO:teuthology.orchestra.run.smithi170.stdout:(32/32): ceph-osd-15.1.1-234.g787db24.el8.x86_6 20 MB/s | 17 MB 00:00 2020-03-19T21:15:48.567 INFO:teuthology.orchestra.run.smithi170.stdout:-------------------------------------------------------------------------------- 2020-03-19T21:15:48.567 INFO:teuthology.orchestra.run.smithi170.stdout:Total 23 MB/s | 29 MB 00:01 2020-03-19T21:15:48.586 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction check 2020-03-19T21:15:48.622 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction check succeeded. 2020-03-19T21:15:48.622 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction test 2020-03-19T21:15:48.978 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction test succeeded. 2020-03-19T21:15:48.978 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction 2020-03-19T21:15:50.167 INFO:teuthology.orchestra.run.smithi170.stdout: Preparing : 1/1 2020-03-19T21:15:50.388 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 1/32 2020-03-19T21:15:50.514 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 2/32 2020-03-19T21:15:50.748 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 3/32 2020-03-19T21:15:50.991 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 4/32 2020-03-19T21:15:51.287 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 5/32 2020-03-19T21:15:51.538 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 6/32 2020-03-19T21:15:51.754 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-webencodings-0.5.1-6.el8.noarch 7/32 2020-03-19T21:15:52.007 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-html5lib-1:0.999999999-6.el8.noarch 8/32 2020-03-19T21:15:52.245 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-waitress-1.2.1-2.el8.1.noarch 9/32 2020-03-19T21:15:52.586 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-cssselect-0.9.2-10.el8.noarch 10/32 2020-03-19T21:15:52.848 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-lxml-4.2.3-1.el8.x86_64 11/32 2020-03-19T21:15:53.068 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 12/32 2020-03-19T21:15:53.302 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 13/32 2020-03-19T21:15:53.534 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 14/32 2020-03-19T21:15:53.702 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 15/32 2020-03-19T21:15:53.917 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-trustme-0.5.2-4.el8.noarch 16/32 2020-03-19T21:15:54.131 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 17/32 2020-03-19T21:15:54.306 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 18/32 2020-03-19T21:15:54.626 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 19/32 2020-03-19T21:15:54.821 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 20/32 2020-03-19T21:15:54.982 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 21/32 2020-03-19T21:15:55.188 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 22/32 2020-03-19T21:15:55.370 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-cheroot-8.2.1-1.el8.noarch 23/32 2020-03-19T21:15:55.573 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 24/32 2020-03-19T21:15:55.849 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-portend-2.6-1.el8.noarch 25/32 2020-03-19T21:15:56.150 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 26/32 2020-03-19T21:15:56.500 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mgr-modules-core-2:15.1.1-234.g787db24.el8.no 27/32 2020-03-19T21:15:56.693 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mgr-2:15.1.1-234.g787db24.el8.x86_64 28/32 2020-03-19T21:15:58.113 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-mgr-2:15.1.1-234.g787db24.el8.x86_64 28/32 2020-03-19T21:15:58.284 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-osd-2:15.1.1-234.g787db24.el8.x86_64 29/32 2020-03-19T21:15:58.672 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-osd-2:15.1.1-234.g787db24.el8.x86_64 29/32 2020-03-19T21:15:58.850 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mon-2:15.1.1-234.g787db24.el8.x86_64 30/32 2020-03-19T21:15:59.045 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-mon-2:15.1.1-234.g787db24.el8.x86_64 30/32 2020-03-19T21:15:59.250 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mds-2:15.1.1-234.g787db24.el8.x86_64 31/32 2020-03-19T21:15:59.270 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-mds-2:15.1.1-234.g787db24.el8.x86_64 31/32 2020-03-19T21:15:59.407 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-2:15.1.1-234.g787db24.el8.x86_64 32/32 2020-03-19T21:15:59.754 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-2:15.1.1-234.g787db24.el8.x86_64 32/32 2020-03-19T21:15:59.754 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-2:15.1.1-234.g787db24.el8.x86_64 1/32 2020-03-19T21:15:59.754 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mds-2:15.1.1-234.g787db24.el8.x86_64 2/32 2020-03-19T21:15:59.754 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mgr-2:15.1.1-234.g787db24.el8.x86_64 3/32 2020-03-19T21:15:59.754 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mon-2:15.1.1-234.g787db24.el8.x86_64 4/32 2020-03-19T21:15:59.754 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-osd-2:15.1.1-234.g787db24.el8.x86_64 5/32 2020-03-19T21:15:59.754 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mgr-modules-core-2:15.1.1-234.g787db24.el8.no 6/32 2020-03-19T21:15:59.755 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-cheroot-8.2.1-1.el8.noarch 7/32 2020-03-19T21:15:59.755 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 8/32 2020-03-19T21:15:59.755 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 9/32 2020-03-19T21:15:59.755 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 10/32 2020-03-19T21:15:59.755 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 11/32 2020-03-19T21:15:59.755 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 12/32 2020-03-19T21:15:59.755 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 13/32 2020-03-19T21:15:59.756 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-portend-2.6-1.el8.noarch 14/32 2020-03-19T21:15:59.756 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 15/32 2020-03-19T21:15:59.756 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 16/32 2020-03-19T21:15:59.756 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 17/32 2020-03-19T21:15:59.756 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-trustme-0.5.2-4.el8.noarch 18/32 2020-03-19T21:15:59.756 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 19/32 2020-03-19T21:15:59.756 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 20/32 2020-03-19T21:15:59.757 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 21/32 2020-03-19T21:15:59.757 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-cssselect-0.9.2-10.el8.noarch 22/32 2020-03-19T21:15:59.757 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-waitress-1.2.1-2.el8.1.noarch 23/32 2020-03-19T21:15:59.757 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 24/32 2020-03-19T21:15:59.757 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 25/32 2020-03-19T21:15:59.757 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-webencodings-0.5.1-6.el8.noarch 26/32 2020-03-19T21:15:59.757 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-lxml-4.2.3-1.el8.x86_64 27/32 2020-03-19T21:15:59.758 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 28/32 2020-03-19T21:15:59.758 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 29/32 2020-03-19T21:15:59.758 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-html5lib-1:0.999999999-6.el8.noarch 30/32 2020-03-19T21:15:59.758 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 31/32 2020-03-19T21:16:02.522 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 32/32 2020-03-19T21:16:02.522 INFO:teuthology.orchestra.run.smithi170.stdout:Installed products updated. 2020-03-19T21:16:02.770 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:16:02.770 INFO:teuthology.orchestra.run.smithi170.stdout:Installed: 2020-03-19T21:16:02.770 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-2:15.1.1-234.g787db24.el8.x86_64 2020-03-19T21:16:02.770 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cssselect-0.9.2-10.el8.noarch 2020-03-19T21:16:02.771 INFO:teuthology.orchestra.run.smithi170.stdout: python3-html5lib-1:0.999999999-6.el8.noarch 2020-03-19T21:16:02.771 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mds-2:15.1.1-234.g787db24.el8.x86_64 2020-03-19T21:16:02.771 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-2:15.1.1-234.g787db24.el8.x86_64 2020-03-19T21:16:02.771 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mon-2:15.1.1-234.g787db24.el8.x86_64 2020-03-19T21:16:02.771 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-osd-2:15.1.1-234.g787db24.el8.x86_64 2020-03-19T21:16:02.771 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-modules-core-2:15.1.1-234.g787db24.el8.noarch 2020-03-19T21:16:02.771 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cheroot-8.2.1-1.el8.noarch 2020-03-19T21:16:02.772 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2020-03-19T21:16:02.772 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco-6.2-6.el8.noarch 2020-03-19T21:16:02.772 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2020-03-19T21:16:02.772 INFO:teuthology.orchestra.run.smithi170.stdout: python3-logutils-0.3.5-11.el8.noarch 2020-03-19T21:16:02.772 INFO:teuthology.orchestra.run.smithi170.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2020-03-19T21:16:02.772 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pecan-1.3.2-9.el8.noarch 2020-03-19T21:16:02.772 INFO:teuthology.orchestra.run.smithi170.stdout: python3-portend-2.6-1.el8.noarch 2020-03-19T21:16:02.772 INFO:teuthology.orchestra.run.smithi170.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2020-03-19T21:16:02.773 INFO:teuthology.orchestra.run.smithi170.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2020-03-19T21:16:02.773 INFO:teuthology.orchestra.run.smithi170.stdout: python3-tempora-1.14.1-5.el8.noarch 2020-03-19T21:16:02.773 INFO:teuthology.orchestra.run.smithi170.stdout: python3-trustme-0.5.2-4.el8.noarch 2020-03-19T21:16:02.773 INFO:teuthology.orchestra.run.smithi170.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2020-03-19T21:16:02.773 INFO:teuthology.orchestra.run.smithi170.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2020-03-19T21:16:02.773 INFO:teuthology.orchestra.run.smithi170.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2020-03-19T21:16:02.773 INFO:teuthology.orchestra.run.smithi170.stdout: python3-waitress-1.2.1-2.el8.1.noarch 2020-03-19T21:16:02.774 INFO:teuthology.orchestra.run.smithi170.stdout: python3-webob-1.8.5-1.el8.1.noarch 2020-03-19T21:16:02.774 INFO:teuthology.orchestra.run.smithi170.stdout: python3-webtest-2.0.33-1.el8.noarch 2020-03-19T21:16:02.774 INFO:teuthology.orchestra.run.smithi170.stdout: python3-webencodings-0.5.1-6.el8.noarch 2020-03-19T21:16:02.774 INFO:teuthology.orchestra.run.smithi170.stdout: python3-lxml-4.2.3-1.el8.x86_64 2020-03-19T21:16:02.774 INFO:teuthology.orchestra.run.smithi170.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2020-03-19T21:16:02.774 INFO:teuthology.orchestra.run.smithi170.stdout: python3-mako-1.0.6-13.el8.noarch 2020-03-19T21:16:02.774 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pytz-2017.2-9.el8.noarch 2020-03-19T21:16:02.774 INFO:teuthology.orchestra.run.smithi170.stdout: python3-markupsafe-0.23-19.el8.x86_64 2020-03-19T21:16:02.775 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:16:02.775 INFO:teuthology.orchestra.run.smithi170.stdout:Complete! 2020-03-19T21:16:02.938 INFO:teuthology.orchestra.run.smithi170:> sudo yum -y install ceph-base 2020-03-19T21:16:03.283 INFO:teuthology.orchestra.run.smithi170.stdout:Updating Subscription Management repositories. 2020-03-19T21:16:03.924 INFO:teuthology.orchestra.run.smithi170.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 79 kB/s | 2.4 kB 00:00 2020-03-19T21:16:03.996 INFO:teuthology.orchestra.run.smithi170.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 91 kB/s | 2.8 kB 00:00 2020-03-19T21:16:04.564 INFO:teuthology.orchestra.run.smithi170.stdout:Package ceph-base-2:15.1.1-234.g787db24.el8.x86_64 is already installed. 2020-03-19T21:16:04.747 INFO:teuthology.orchestra.run.smithi170.stdout:Dependencies resolved. 2020-03-19T21:16:04.748 INFO:teuthology.orchestra.run.smithi170.stdout:Nothing to do. 2020-03-19T21:16:04.748 INFO:teuthology.orchestra.run.smithi170.stdout:Complete! 2020-03-19T21:16:04.903 INFO:teuthology.orchestra.run.smithi170:> sudo yum -y install cephadm 2020-03-19T21:16:05.243 INFO:teuthology.orchestra.run.smithi170.stdout:Updating Subscription Management repositories. 2020-03-19T21:16:05.852 INFO:teuthology.orchestra.run.smithi170.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 78 kB/s | 2.4 kB 00:00 2020-03-19T21:16:05.922 INFO:teuthology.orchestra.run.smithi170.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 92 kB/s | 2.8 kB 00:00 2020-03-19T21:16:06.651 INFO:teuthology.orchestra.run.smithi170.stdout:Dependencies resolved. 2020-03-19T21:16:06.652 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2020-03-19T21:16:06.652 INFO:teuthology.orchestra.run.smithi170.stdout: Package Architecture Version Repository Size 2020-03-19T21:16:06.652 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2020-03-19T21:16:06.652 INFO:teuthology.orchestra.run.smithi170.stdout:Installing: 2020-03-19T21:16:06.653 INFO:teuthology.orchestra.run.smithi170.stdout: cephadm x86_64 2:15.1.1-234.g787db24.el8 ceph 39 k 2020-03-19T21:16:06.653 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:16:06.653 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction Summary 2020-03-19T21:16:06.653 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2020-03-19T21:16:06.653 INFO:teuthology.orchestra.run.smithi170.stdout:Install 1 Package 2020-03-19T21:16:06.653 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:16:06.654 INFO:teuthology.orchestra.run.smithi170.stdout:Total download size: 39 k 2020-03-19T21:16:06.654 INFO:teuthology.orchestra.run.smithi170.stdout:Installed size: 148 k 2020-03-19T21:16:06.654 INFO:teuthology.orchestra.run.smithi170.stdout:Downloading Packages: 2020-03-19T21:16:07.019 INFO:teuthology.orchestra.run.smithi170.stdout:cephadm-15.1.1-234.g787db24.el8.x86_64.rpm 108 kB/s | 39 kB 00:00 2020-03-19T21:16:07.020 INFO:teuthology.orchestra.run.smithi170.stdout:-------------------------------------------------------------------------------- 2020-03-19T21:16:07.020 INFO:teuthology.orchestra.run.smithi170.stdout:Total 107 kB/s | 39 kB 00:00 2020-03-19T21:16:07.022 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction check 2020-03-19T21:16:07.029 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction check succeeded. 2020-03-19T21:16:07.030 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction test 2020-03-19T21:16:07.031 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction test succeeded. 2020-03-19T21:16:07.031 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction 2020-03-19T21:16:07.082 INFO:teuthology.orchestra.run.smithi170.stdout: Preparing : 1/1 2020-03-19T21:16:08.413 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: cephadm-2:15.1.1-234.g787db24.el8.x86_64 1/1 2020-03-19T21:16:08.686 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : cephadm-2:15.1.1-234.g787db24.el8.x86_64 1/1 2020-03-19T21:16:10.562 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : cephadm-2:15.1.1-234.g787db24.el8.x86_64 1/1 2020-03-19T21:16:10.563 INFO:teuthology.orchestra.run.smithi170.stdout:Installed products updated. 2020-03-19T21:16:10.751 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:16:10.752 INFO:teuthology.orchestra.run.smithi170.stdout:Installed: 2020-03-19T21:16:10.752 INFO:teuthology.orchestra.run.smithi170.stdout: cephadm-2:15.1.1-234.g787db24.el8.x86_64 2020-03-19T21:16:10.752 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:16:10.752 INFO:teuthology.orchestra.run.smithi170.stdout:Complete! 2020-03-19T21:16:10.916 INFO:teuthology.orchestra.run.smithi170:> sudo yum -y install ceph-immutable-object-cache 2020-03-19T21:16:11.265 INFO:teuthology.orchestra.run.smithi170.stdout:Updating Subscription Management repositories. 2020-03-19T21:16:11.950 INFO:teuthology.orchestra.run.smithi170.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 78 kB/s | 2.4 kB 00:00 2020-03-19T21:16:12.023 INFO:teuthology.orchestra.run.smithi170.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 92 kB/s | 2.8 kB 00:00 2020-03-19T21:16:13.514 INFO:teuthology.orchestra.run.smithi170.stdout:Dependencies resolved. 2020-03-19T21:16:13.515 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2020-03-19T21:16:13.515 INFO:teuthology.orchestra.run.smithi170.stdout: Package Arch Version Repo Size 2020-03-19T21:16:13.515 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2020-03-19T21:16:13.515 INFO:teuthology.orchestra.run.smithi170.stdout:Installing: 2020-03-19T21:16:13.516 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-immutable-object-cache x86_64 2:15.1.1-234.g787db24.el8 ceph 203 k 2020-03-19T21:16:13.516 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:16:13.516 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction Summary 2020-03-19T21:16:13.516 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2020-03-19T21:16:13.516 INFO:teuthology.orchestra.run.smithi170.stdout:Install 1 Package 2020-03-19T21:16:13.516 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:16:13.517 INFO:teuthology.orchestra.run.smithi170.stdout:Total download size: 203 k 2020-03-19T21:16:13.517 INFO:teuthology.orchestra.run.smithi170.stdout:Installed size: 743 k 2020-03-19T21:16:13.517 INFO:teuthology.orchestra.run.smithi170.stdout:Downloading Packages: 2020-03-19T21:16:13.802 INFO:teuthology.orchestra.run.smithi170.stdout:ceph-immutable-object-cache-15.1.1-234.g787db24 712 kB/s | 203 kB 00:00 2020-03-19T21:16:13.802 INFO:teuthology.orchestra.run.smithi170.stdout:-------------------------------------------------------------------------------- 2020-03-19T21:16:13.803 INFO:teuthology.orchestra.run.smithi170.stdout:Total 709 kB/s | 203 kB 00:00 2020-03-19T21:16:13.805 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction check 2020-03-19T21:16:13.817 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction check succeeded. 2020-03-19T21:16:13.817 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction test 2020-03-19T21:16:13.867 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction test succeeded. 2020-03-19T21:16:13.868 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction 2020-03-19T21:16:13.993 INFO:teuthology.orchestra.run.smithi170.stdout: Preparing : 1/1 2020-03-19T21:16:14.140 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-immutable-object-cache-2:15.1.1-234.g787db24.el 1/1 2020-03-19T21:16:14.474 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-immutable-object-cache-2:15.1.1-234.g787db24.el 1/1 2020-03-19T21:16:16.548 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-immutable-object-cache-2:15.1.1-234.g787db24.el 1/1 2020-03-19T21:16:16.549 INFO:teuthology.orchestra.run.smithi170.stdout:Installed products updated. 2020-03-19T21:16:16.750 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:16:16.751 INFO:teuthology.orchestra.run.smithi170.stdout:Installed: 2020-03-19T21:16:16.751 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-immutable-object-cache-2:15.1.1-234.g787db24.el8.x86_64 2020-03-19T21:16:16.751 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:16:16.751 INFO:teuthology.orchestra.run.smithi170.stdout:Complete! 2020-03-19T21:16:16.925 INFO:teuthology.orchestra.run.smithi170:> sudo yum -y install ceph-mgr 2020-03-19T21:16:17.266 INFO:teuthology.orchestra.run.smithi170.stdout:Updating Subscription Management repositories. 2020-03-19T21:16:17.934 INFO:teuthology.orchestra.run.smithi170.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 79 kB/s | 2.4 kB 00:00 2020-03-19T21:16:18.007 INFO:teuthology.orchestra.run.smithi170.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 88 kB/s | 2.8 kB 00:00 2020-03-19T21:16:18.584 INFO:teuthology.orchestra.run.smithi170.stdout:Package ceph-mgr-2:15.1.1-234.g787db24.el8.x86_64 is already installed. 2020-03-19T21:16:18.764 INFO:teuthology.orchestra.run.smithi170.stdout:Dependencies resolved. 2020-03-19T21:16:18.765 INFO:teuthology.orchestra.run.smithi170.stdout:Nothing to do. 2020-03-19T21:16:18.765 INFO:teuthology.orchestra.run.smithi170.stdout:Complete! 2020-03-19T21:16:18.924 INFO:teuthology.orchestra.run.smithi170:> sudo yum -y install ceph-mgr-dashboard 2020-03-19T21:16:19.264 INFO:teuthology.orchestra.run.smithi170.stdout:Updating Subscription Management repositories. 2020-03-19T21:16:19.918 INFO:teuthology.orchestra.run.smithi170.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 76 kB/s | 2.4 kB 00:00 2020-03-19T21:16:20.722 INFO:teuthology.orchestra.run.smithi170.stdout:Dependencies resolved. 2020-03-19T21:16:20.724 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2020-03-19T21:16:20.724 INFO:teuthology.orchestra.run.smithi170.stdout: Package ArchVersion Repository Size 2020-03-19T21:16:20.724 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2020-03-19T21:16:20.724 INFO:teuthology.orchestra.run.smithi170.stdout:Installing: 2020-03-19T21:16:20.724 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-dashboard noarch2:15.1.1-234.g787db24.el8 ceph-noarch 3.3 M 2020-03-19T21:16:20.724 INFO:teuthology.orchestra.run.smithi170.stdout:Installing dependencies: 2020-03-19T21:16:20.724 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-grafana-dashboards noarch2:15.1.1-234.g787db24.el8 ceph-noarch 22 k 2020-03-19T21:16:20.725 INFO:teuthology.orchestra.run.smithi170.stdout: python3-repoze-lru noarch0.7-6.el8 copr:copr.fedorainfracloud.org:ktdreyer:ceph-el8 34 k 2020-03-19T21:16:20.725 INFO:teuthology.orchestra.run.smithi170.stdout: python3-routes noarch2.4.1-12.el8 copr:copr.fedorainfracloud.org:ktdreyer:ceph-el8 197 k 2020-03-19T21:16:20.725 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jwt noarch1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2020-03-19T21:16:20.725 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:16:20.725 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction Summary 2020-03-19T21:16:20.725 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2020-03-19T21:16:20.725 INFO:teuthology.orchestra.run.smithi170.stdout:Install 5 Packages 2020-03-19T21:16:20.726 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:16:20.726 INFO:teuthology.orchestra.run.smithi170.stdout:Total download size: 3.6 M 2020-03-19T21:16:20.726 INFO:teuthology.orchestra.run.smithi170.stdout:Installed size: 18 M 2020-03-19T21:16:20.726 INFO:teuthology.orchestra.run.smithi170.stdout:Downloading Packages: 2020-03-19T21:16:20.768 INFO:teuthology.orchestra.run.smithi170.stdout:(1/5): python3-repoze-lru-0.7-6.el8.noarch.rpm 797 kB/s | 34 kB 00:00 2020-03-19T21:16:20.792 INFO:teuthology.orchestra.run.smithi170.stdout:(2/5): python3-routes-2.4.1-12.el8.noarch.rpm 8.1 MB/s | 197 kB 00:00 2020-03-19T21:16:20.825 INFO:teuthology.orchestra.run.smithi170.stdout:(3/5): python3-jwt-1.6.1-2.el8.noarch.rpm 1.3 MB/s | 43 kB 00:00 2020-03-19T21:16:20.892 INFO:teuthology.orchestra.run.smithi170.stdout:(4/5): ceph-grafana-dashboards-15.1.1-234.g787d 134 kB/s | 22 kB 00:00 2020-03-19T21:16:21.189 INFO:teuthology.orchestra.run.smithi170.stdout:(5/5): ceph-mgr-dashboard-15.1.1-234.g787db24.e 7.2 MB/s | 3.3 MB 00:00 2020-03-19T21:16:21.190 INFO:teuthology.orchestra.run.smithi170.stdout:-------------------------------------------------------------------------------- 2020-03-19T21:16:21.191 INFO:teuthology.orchestra.run.smithi170.stdout:Total 7.8 MB/s | 3.6 MB 00:00 2020-03-19T21:16:21.196 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction check 2020-03-19T21:16:21.204 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction check succeeded. 2020-03-19T21:16:21.204 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction test 2020-03-19T21:16:21.353 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction test succeeded. 2020-03-19T21:16:21.353 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction 2020-03-19T21:16:21.611 INFO:teuthology.orchestra.run.smithi170.stdout: Preparing : 1/1 2020-03-19T21:16:21.823 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 1/5 2020-03-19T21:16:22.054 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 2/5 2020-03-19T21:16:22.270 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 3/5 2020-03-19T21:16:23.286 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-grafana-dashboards-2:15.1.1-234.g787db24.el8.no 4/5 2020-03-19T21:16:23.538 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mgr-dashboard-2:15.1.1-234.g787db24.el8.noarch 5/5 2020-03-19T21:16:23.783 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-mgr-dashboard-2:15.1.1-234.g787db24.el8.noarch 5/5 2020-03-19T21:16:23.783 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-grafana-dashboards-2:15.1.1-234.g787db24.el8.no 1/5 2020-03-19T21:16:23.783 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mgr-dashboard-2:15.1.1-234.g787db24.el8.noarch 2/5 2020-03-19T21:16:23.783 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 3/5 2020-03-19T21:16:23.783 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 4/5 2020-03-19T21:16:26.249 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 5/5 2020-03-19T21:16:26.249 INFO:teuthology.orchestra.run.smithi170.stdout:Installed products updated. 2020-03-19T21:16:26.450 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:16:26.450 INFO:teuthology.orchestra.run.smithi170.stdout:Installed: 2020-03-19T21:16:26.450 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-dashboard-2:15.1.1-234.g787db24.el8.noarch 2020-03-19T21:16:26.450 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-grafana-dashboards-2:15.1.1-234.g787db24.el8.noarch 2020-03-19T21:16:26.450 INFO:teuthology.orchestra.run.smithi170.stdout: python3-repoze-lru-0.7-6.el8.noarch 2020-03-19T21:16:26.450 INFO:teuthology.orchestra.run.smithi170.stdout: python3-routes-2.4.1-12.el8.noarch 2020-03-19T21:16:26.451 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jwt-1.6.1-2.el8.noarch 2020-03-19T21:16:26.451 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:16:26.451 INFO:teuthology.orchestra.run.smithi170.stdout:Complete! 2020-03-19T21:16:26.615 INFO:teuthology.orchestra.run.smithi170:> sudo yum -y install ceph-mgr-diskprediction-local 2020-03-19T21:16:26.961 INFO:teuthology.orchestra.run.smithi170.stdout:Updating Subscription Management repositories. 2020-03-19T21:16:27.740 INFO:teuthology.orchestra.run.smithi170.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 79 kB/s | 2.4 kB 00:00 2020-03-19T21:16:27.811 INFO:teuthology.orchestra.run.smithi170.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 89 kB/s | 2.8 kB 00:00 2020-03-19T21:16:28.569 INFO:teuthology.orchestra.run.smithi170.stdout:Dependencies resolved. 2020-03-19T21:16:28.571 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2020-03-19T21:16:28.571 INFO:teuthology.orchestra.run.smithi170.stdout: Package ArchVersion Repository Size 2020-03-19T21:16:28.572 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2020-03-19T21:16:28.572 INFO:teuthology.orchestra.run.smithi170.stdout:Installing: 2020-03-19T21:16:28.572 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-diskprediction-local noarch2:15.1.1-234.g787db24.el8 ceph-noarch 8.6 M 2020-03-19T21:16:28.572 INFO:teuthology.orchestra.run.smithi170.stdout:Installing dependencies: 2020-03-19T21:16:28.572 INFO:teuthology.orchestra.run.smithi170.stdout: libquadmath x86_648.3.1-4.5.el8 rhel-8-for-x86_64-baseos-rpms 169 k 2020-03-19T21:16:28.573 INFO:teuthology.orchestra.run.smithi170.stdout: libgfortran x86_648.3.1-4.5.el8 rhel-8-for-x86_64-baseos-rpms 638 k 2020-03-19T21:16:28.573 INFO:teuthology.orchestra.run.smithi170.stdout: openblas-threads x86_640.3.3-2.el8 rhel-8-for-x86_64-appstream-rpms 4.4 M 2020-03-19T21:16:28.573 INFO:teuthology.orchestra.run.smithi170.stdout: openblas x86_640.3.3-2.el8 rhel-8-for-x86_64-appstream-rpms 4.3 M 2020-03-19T21:16:28.573 INFO:teuthology.orchestra.run.smithi170.stdout: python3-numpy x86_641:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2020-03-19T21:16:28.573 INFO:teuthology.orchestra.run.smithi170.stdout: python3-numpy-f2py x86_641:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2020-03-19T21:16:28.573 INFO:teuthology.orchestra.run.smithi170.stdout: python3-scipy x86_641.0.0-20.module+el8.1.0+3334+5cb623d7 rhel-8-for-x86_64-appstream-rpms 15 M 2020-03-19T21:16:28.573 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:16:28.574 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction Summary 2020-03-19T21:16:28.574 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2020-03-19T21:16:28.574 INFO:teuthology.orchestra.run.smithi170.stdout:Install 8 Packages 2020-03-19T21:16:28.574 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:16:28.574 INFO:teuthology.orchestra.run.smithi170.stdout:Total download size: 37 M 2020-03-19T21:16:28.574 INFO:teuthology.orchestra.run.smithi170.stdout:Installed size: 204 M 2020-03-19T21:16:28.574 INFO:teuthology.orchestra.run.smithi170.stdout:Downloading Packages: 2020-03-19T21:16:28.614 INFO:teuthology.orchestra.run.smithi170.stdout:(1/8): libquadmath-8.3.1-4.5.el8.x86_64.rpm 3.9 MB/s | 169 kB 00:00 2020-03-19T21:16:28.627 INFO:teuthology.orchestra.run.smithi170.stdout:(2/8): libgfortran-8.3.1-4.5.el8.x86_64.rpm 11 MB/s | 638 kB 00:00 2020-03-19T21:16:28.672 INFO:teuthology.orchestra.run.smithi170.stdout:(3/8): openblas-threads-0.3.3-2.el8.x86_64.rpm 76 MB/s | 4.4 MB 00:00 2020-03-19T21:16:28.716 INFO:teuthology.orchestra.run.smithi170.stdout:(4/8): openblas-0.3.3-2.el8.x86_64.rpm 49 MB/s | 4.3 MB 00:00 2020-03-19T21:16:28.740 INFO:teuthology.orchestra.run.smithi170.stdout:(5/8): python3-numpy-1.14.3-9.el8.x86_64.rpm 55 MB/s | 3.7 MB 00:00 2020-03-19T21:16:28.742 INFO:teuthology.orchestra.run.smithi170.stdout:(6/8): python3-numpy-f2py-1.14.3-9.el8.x86_64.r 8.4 MB/s | 225 kB 00:00 2020-03-19T21:16:28.909 INFO:teuthology.orchestra.run.smithi170.stdout:(7/8): python3-scipy-1.0.0-20.module+el8.1.0+33 86 MB/s | 15 MB 00:00 2020-03-19T21:16:29.780 INFO:teuthology.orchestra.run.smithi170.stdout:(8/8): ceph-mgr-diskprediction-local-15.1.1-234 7.1 MB/s | 8.6 MB 00:01 2020-03-19T21:16:29.781 INFO:teuthology.orchestra.run.smithi170.stdout:-------------------------------------------------------------------------------- 2020-03-19T21:16:29.782 INFO:teuthology.orchestra.run.smithi170.stdout:Total 30 MB/s | 37 MB 00:01 2020-03-19T21:16:29.795 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction check 2020-03-19T21:16:29.815 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction check succeeded. 2020-03-19T21:16:29.815 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction test 2020-03-19T21:16:30.116 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction test succeeded. 2020-03-19T21:16:30.116 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction 2020-03-19T21:16:30.668 INFO:teuthology.orchestra.run.smithi170.stdout: Preparing : 1/1 2020-03-19T21:16:30.862 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libquadmath-8.3.1-4.5.el8.x86_64 1/8 2020-03-19T21:16:30.997 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: libquadmath-8.3.1-4.5.el8.x86_64 1/8 2020-03-19T21:16:31.153 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libgfortran-8.3.1-4.5.el8.x86_64 2/8 2020-03-19T21:16:31.681 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: libgfortran-8.3.1-4.5.el8.x86_64 2/8 2020-03-19T21:16:31.811 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : openblas-threads-0.3.3-2.el8.x86_64 3/8 2020-03-19T21:16:32.357 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: openblas-threads-0.3.3-2.el8.x86_64 3/8 2020-03-19T21:16:32.553 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : openblas-0.3.3-2.el8.x86_64 4/8 2020-03-19T21:16:33.272 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: openblas-0.3.3-2.el8.x86_64 4/8 2020-03-19T21:16:33.608 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-numpy-1:1.14.3-9.el8.x86_64 5/8 2020-03-19T21:16:33.836 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 6/8 2020-03-19T21:16:35.807 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 6/8 2020-03-19T21:16:37.133 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb623d7. 7/8 2020-03-19T21:16:37.353 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mgr-diskprediction-local-2:15.1.1-234.g787db24. 8/8 2020-03-19T21:16:37.597 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:15.1.1-234.g787db24. 8/8 2020-03-19T21:16:37.597 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mgr-diskprediction-local-2:15.1.1-234.g787db24. 1/8 2020-03-19T21:16:37.597 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libquadmath-8.3.1-4.5.el8.x86_64 2/8 2020-03-19T21:16:37.597 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libgfortran-8.3.1-4.5.el8.x86_64 3/8 2020-03-19T21:16:37.597 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : openblas-threads-0.3.3-2.el8.x86_64 4/8 2020-03-19T21:16:37.597 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : openblas-0.3.3-2.el8.x86_64 5/8 2020-03-19T21:16:37.598 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-numpy-1:1.14.3-9.el8.x86_64 6/8 2020-03-19T21:16:37.598 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 7/8 2020-03-19T21:16:39.693 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb623d7. 8/8 2020-03-19T21:16:39.693 INFO:teuthology.orchestra.run.smithi170.stdout:Installed products updated. 2020-03-19T21:16:39.910 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:16:39.910 INFO:teuthology.orchestra.run.smithi170.stdout:Installed: 2020-03-19T21:16:39.910 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-diskprediction-local-2:15.1.1-234.g787db24.el8.noarch 2020-03-19T21:16:39.910 INFO:teuthology.orchestra.run.smithi170.stdout: libquadmath-8.3.1-4.5.el8.x86_64 2020-03-19T21:16:39.910 INFO:teuthology.orchestra.run.smithi170.stdout: libgfortran-8.3.1-4.5.el8.x86_64 2020-03-19T21:16:39.911 INFO:teuthology.orchestra.run.smithi170.stdout: openblas-threads-0.3.3-2.el8.x86_64 2020-03-19T21:16:39.911 INFO:teuthology.orchestra.run.smithi170.stdout: openblas-0.3.3-2.el8.x86_64 2020-03-19T21:16:39.911 INFO:teuthology.orchestra.run.smithi170.stdout: python3-numpy-1:1.14.3-9.el8.x86_64 2020-03-19T21:16:39.911 INFO:teuthology.orchestra.run.smithi170.stdout: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 2020-03-19T21:16:39.911 INFO:teuthology.orchestra.run.smithi170.stdout: python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb623d7.x86_64 2020-03-19T21:16:39.911 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:16:39.912 INFO:teuthology.orchestra.run.smithi170.stdout:Complete! 2020-03-19T21:16:40.364 INFO:teuthology.orchestra.run.smithi170:> sudo yum -y install ceph-mgr-rook 2020-03-19T21:16:40.727 INFO:teuthology.orchestra.run.smithi170.stdout:Updating Subscription Management repositories. 2020-03-19T21:16:41.384 INFO:teuthology.orchestra.run.smithi170.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 80 kB/s | 2.4 kB 00:00 2020-03-19T21:16:41.458 INFO:teuthology.orchestra.run.smithi170.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 84 kB/s | 2.8 kB 00:00 2020-03-19T21:16:42.209 INFO:teuthology.orchestra.run.smithi170.stdout:Dependencies resolved. 2020-03-19T21:16:42.211 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2020-03-19T21:16:42.211 INFO:teuthology.orchestra.run.smithi170.stdout: Package ArchVersion Repository Size 2020-03-19T21:16:42.211 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2020-03-19T21:16:42.211 INFO:teuthology.orchestra.run.smithi170.stdout:Installing: 2020-03-19T21:16:42.211 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-rook noarch2:15.1.1-234.g787db24.el8 ceph-noarch 25 k 2020-03-19T21:16:42.212 INFO:teuthology.orchestra.run.smithi170.stdout:Installing dependencies: 2020-03-19T21:16:42.212 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cachetools noarch3.1.1-4.el8 epel 33 k 2020-03-19T21:16:42.212 INFO:teuthology.orchestra.run.smithi170.stdout: python3-certifi noarch2018.10.15-7.el8 epel 16 k 2020-03-19T21:16:42.212 INFO:teuthology.orchestra.run.smithi170.stdout: python3-google-auth noarch1:1.1.1-10.el8 epel 81 k 2020-03-19T21:16:42.212 INFO:teuthology.orchestra.run.smithi170.stdout: python3-kubernetes noarch10.0.1-2.el8 epel 1.1 M 2020-03-19T21:16:42.212 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rsa noarch3.4.2-12.el8 epel 73 k 2020-03-19T21:16:42.212 INFO:teuthology.orchestra.run.smithi170.stdout: python3-websocket-client noarch0.56.0-5.el8 epel 61 k 2020-03-19T21:16:42.213 INFO:teuthology.orchestra.run.smithi170.stdout: python3-requests-oauthlib noarch1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2020-03-19T21:16:42.213 INFO:teuthology.orchestra.run.smithi170.stdout: python3-oauthlib noarch2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2020-03-19T21:16:42.213 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jsonpatch noarch1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2020-03-19T21:16:42.213 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jsonpointer noarch1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2020-03-19T21:16:42.213 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pyasn1 noarch0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2020-03-19T21:16:42.214 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pyasn1-modules noarch0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2020-03-19T21:16:42.214 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:16:42.214 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction Summary 2020-03-19T21:16:42.214 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2020-03-19T21:16:42.215 INFO:teuthology.orchestra.run.smithi170.stdout:Install 13 Packages 2020-03-19T21:16:42.215 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:16:42.216 INFO:teuthology.orchestra.run.smithi170.stdout:Total download size: 1.9 M 2020-03-19T21:16:42.216 INFO:teuthology.orchestra.run.smithi170.stdout:Installed size: 23 M 2020-03-19T21:16:42.216 INFO:teuthology.orchestra.run.smithi170.stdout:Downloading Packages: 2020-03-19T21:16:42.261 INFO:teuthology.orchestra.run.smithi170.stdout:(1/13): python3-certifi-2018.10.15-7.el8.noarch 348 kB/s | 16 kB 00:00 2020-03-19T21:16:42.261 INFO:teuthology.orchestra.run.smithi170.stdout:(2/13): python3-cachetools-3.1.1-4.el8.noarch.r 698 kB/s | 33 kB 00:00 2020-03-19T21:16:42.292 INFO:teuthology.orchestra.run.smithi170.stdout:(3/13): python3-google-auth-1.1.1-10.el8.noarch 2.6 MB/s | 81 kB 00:00 2020-03-19T21:16:42.308 INFO:teuthology.orchestra.run.smithi170.stdout:(4/13): python3-rsa-3.4.2-12.el8.noarch.rpm 4.4 MB/s | 73 kB 00:00 2020-03-19T21:16:42.324 INFO:teuthology.orchestra.run.smithi170.stdout:(5/13): python3-websocket-client-0.56.0-5.el8.n 3.8 MB/s | 61 kB 00:00 2020-03-19T21:16:42.365 INFO:teuthology.orchestra.run.smithi170.stdout:(6/13): python3-requests-oauthlib-1.0.0-1.el8.n 1.0 MB/s | 43 kB 00:00 2020-03-19T21:16:42.370 INFO:teuthology.orchestra.run.smithi170.stdout:(7/13): python3-kubernetes-10.0.1-2.el8.noarch. 10 MB/s | 1.1 MB 00:00 2020-03-19T21:16:42.382 INFO:teuthology.orchestra.run.smithi170.stdout:(8/13): ceph-mgr-rook-15.1.1-234.g787db24.el8.n 151 kB/s | 25 kB 00:00 2020-03-19T21:16:42.408 INFO:teuthology.orchestra.run.smithi170.stdout:(9/13): python3-oauthlib-2.1.0-1.el8.noarch.rpm 3.9 MB/s | 155 kB 00:00 2020-03-19T21:16:42.426 INFO:teuthology.orchestra.run.smithi170.stdout:(10/13): python3-jsonpatch-1.21-2.el8.noarch.rp 476 kB/s | 27 kB 00:00 2020-03-19T21:16:42.436 INFO:teuthology.orchestra.run.smithi170.stdout:(11/13): python3-jsonpointer-1.10-11.el8.noarch 370 kB/s | 20 kB 00:00 2020-03-19T21:16:42.440 INFO:teuthology.orchestra.run.smithi170.stdout:(12/13): python3-pyasn1-0.3.7-6.el8.noarch.rpm 3.4 MB/s | 126 kB 00:00 2020-03-19T21:16:42.458 INFO:teuthology.orchestra.run.smithi170.stdout:(13/13): python3-pyasn1-modules-0.3.7-6.el8.noa 3.3 MB/s | 110 kB 00:00 2020-03-19T21:16:42.458 INFO:teuthology.orchestra.run.smithi170.stdout:-------------------------------------------------------------------------------- 2020-03-19T21:16:42.459 INFO:teuthology.orchestra.run.smithi170.stdout:Total 7.6 MB/s | 1.9 MB 00:00 2020-03-19T21:16:42.466 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction check 2020-03-19T21:16:42.480 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction check succeeded. 2020-03-19T21:16:42.480 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction test 2020-03-19T21:16:42.548 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction test succeeded. 2020-03-19T21:16:42.548 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction 2020-03-19T21:16:43.057 INFO:teuthology.orchestra.run.smithi170.stdout: Preparing : 1/1 2020-03-19T21:16:43.325 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 1/13 2020-03-19T21:16:43.589 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-rsa-3.4.2-12.el8.noarch 2/13 2020-03-19T21:16:43.808 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 3/13 2020-03-19T21:16:43.992 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 4/13 2020-03-19T21:16:44.259 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 5/13 2020-03-19T21:16:44.481 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 6/13 2020-03-19T21:16:44.688 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 7/13 2020-03-19T21:16:44.908 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 8/13 2020-03-19T21:16:45.124 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 9/13 2020-03-19T21:16:45.365 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 10/13 2020-03-19T21:16:46.509 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 11/13 2020-03-19T21:16:46.801 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-kubernetes-10.0.1-2.el8.noarch 12/13 2020-03-19T21:16:46.967 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mgr-rook-2:15.1.1-234.g787db24.el8.noarch 13/13 2020-03-19T21:16:47.187 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-mgr-rook-2:15.1.1-234.g787db24.el8.noarch 13/13 2020-03-19T21:16:47.187 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mgr-rook-2:15.1.1-234.g787db24.el8.noarch 1/13 2020-03-19T21:16:47.188 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 2/13 2020-03-19T21:16:47.188 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 3/13 2020-03-19T21:16:47.188 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 4/13 2020-03-19T21:16:47.188 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-kubernetes-10.0.1-2.el8.noarch 5/13 2020-03-19T21:16:47.188 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-rsa-3.4.2-12.el8.noarch 6/13 2020-03-19T21:16:47.188 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 7/13 2020-03-19T21:16:47.189 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 8/13 2020-03-19T21:16:47.189 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 9/13 2020-03-19T21:16:47.189 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 10/13 2020-03-19T21:16:47.189 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 11/13 2020-03-19T21:16:47.189 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 12/13 2020-03-19T21:16:49.194 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 13/13 2020-03-19T21:16:49.194 INFO:teuthology.orchestra.run.smithi170.stdout:Installed products updated. 2020-03-19T21:16:49.403 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:16:49.403 INFO:teuthology.orchestra.run.smithi170.stdout:Installed: 2020-03-19T21:16:49.404 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-rook-2:15.1.1-234.g787db24.el8.noarch 2020-03-19T21:16:49.404 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cachetools-3.1.1-4.el8.noarch 2020-03-19T21:16:49.404 INFO:teuthology.orchestra.run.smithi170.stdout: python3-certifi-2018.10.15-7.el8.noarch 2020-03-19T21:16:49.404 INFO:teuthology.orchestra.run.smithi170.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2020-03-19T21:16:49.404 INFO:teuthology.orchestra.run.smithi170.stdout: python3-kubernetes-10.0.1-2.el8.noarch 2020-03-19T21:16:49.404 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rsa-3.4.2-12.el8.noarch 2020-03-19T21:16:49.404 INFO:teuthology.orchestra.run.smithi170.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2020-03-19T21:16:49.405 INFO:teuthology.orchestra.run.smithi170.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2020-03-19T21:16:49.405 INFO:teuthology.orchestra.run.smithi170.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2020-03-19T21:16:49.405 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jsonpatch-1.21-2.el8.noarch 2020-03-19T21:16:49.405 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jsonpointer-1.10-11.el8.noarch 2020-03-19T21:16:49.405 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2020-03-19T21:16:49.405 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2020-03-19T21:16:49.405 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:16:49.406 INFO:teuthology.orchestra.run.smithi170.stdout:Complete! 2020-03-19T21:16:49.568 INFO:teuthology.orchestra.run.smithi170:> sudo yum -y install ceph-mgr-cephadm 2020-03-19T21:16:49.920 INFO:teuthology.orchestra.run.smithi170.stdout:Updating Subscription Management repositories. 2020-03-19T21:16:50.528 INFO:teuthology.orchestra.run.smithi170.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 82 kB/s | 2.4 kB 00:00 2020-03-19T21:16:50.599 INFO:teuthology.orchestra.run.smithi170.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 96 kB/s | 2.8 kB 00:00 2020-03-19T21:16:51.374 INFO:teuthology.orchestra.run.smithi170.stdout:Dependencies resolved. 2020-03-19T21:16:51.375 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2020-03-19T21:16:51.375 INFO:teuthology.orchestra.run.smithi170.stdout: Package ArchVersion Repository Size 2020-03-19T21:16:51.375 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2020-03-19T21:16:51.375 INFO:teuthology.orchestra.run.smithi170.stdout:Installing: 2020-03-19T21:16:51.375 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-cephadm noarch2:15.1.1-234.g787db24.el8 ceph-noarch 38 k 2020-03-19T21:16:51.376 INFO:teuthology.orchestra.run.smithi170.stdout:Installing dependencies: 2020-03-19T21:16:51.376 INFO:teuthology.orchestra.run.smithi170.stdout: python3-execnet noarch1.7.1-1.el8.2 copr:copr.fedorainfracloud.org:ktdreyer:ceph-el8 356 k 2020-03-19T21:16:51.376 INFO:teuthology.orchestra.run.smithi170.stdout: python3-remoto noarch1.1.4-4.el8 copr:copr.fedorainfracloud.org:ktdreyer:ceph-el8 48 k 2020-03-19T21:16:51.376 INFO:teuthology.orchestra.run.smithi170.stdout: python3-apipkg noarch1.5-6.el8 epel 21 k 2020-03-19T21:16:51.376 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:16:51.376 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction Summary 2020-03-19T21:16:51.376 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2020-03-19T21:16:51.377 INFO:teuthology.orchestra.run.smithi170.stdout:Install 4 Packages 2020-03-19T21:16:51.377 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:16:51.377 INFO:teuthology.orchestra.run.smithi170.stdout:Total download size: 463 k 2020-03-19T21:16:51.377 INFO:teuthology.orchestra.run.smithi170.stdout:Installed size: 1.4 M 2020-03-19T21:16:51.378 INFO:teuthology.orchestra.run.smithi170.stdout:Downloading Packages: 2020-03-19T21:16:51.431 INFO:teuthology.orchestra.run.smithi170.stdout:(1/4): python3-remoto-1.1.4-4.el8.noarch.rpm 896 kB/s | 48 kB 00:00 2020-03-19T21:16:51.442 INFO:teuthology.orchestra.run.smithi170.stdout:(2/4): python3-execnet-1.7.1-1.el8.2.noarch.rpm 5.3 MB/s | 356 kB 00:00 2020-03-19T21:16:51.488 INFO:teuthology.orchestra.run.smithi170.stdout:(3/4): python3-apipkg-1.5-6.el8.noarch.rpm 372 kB/s | 21 kB 00:00 2020-03-19T21:16:51.545 INFO:teuthology.orchestra.run.smithi170.stdout:(4/4): ceph-mgr-cephadm-15.1.1-234.g787db24.el8 224 kB/s | 38 kB 00:00 2020-03-19T21:16:51.546 INFO:teuthology.orchestra.run.smithi170.stdout:-------------------------------------------------------------------------------- 2020-03-19T21:16:51.547 INFO:teuthology.orchestra.run.smithi170.stdout:Total 2.6 MB/s | 463 kB 00:00 2020-03-19T21:16:51.551 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction check 2020-03-19T21:16:51.558 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction check succeeded. 2020-03-19T21:16:51.558 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction test 2020-03-19T21:16:51.585 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction test succeeded. 2020-03-19T21:16:51.585 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction 2020-03-19T21:16:51.761 INFO:teuthology.orchestra.run.smithi170.stdout: Preparing : 1/1 2020-03-19T21:16:52.085 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-apipkg-1.5-6.el8.noarch 1/4 2020-03-19T21:16:52.396 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-execnet-1.7.1-1.el8.2.noarch 2/4 2020-03-19T21:16:52.647 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-remoto-1.1.4-4.el8.noarch 3/4 2020-03-19T21:16:52.808 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mgr-cephadm-2:15.1.1-234.g787db24.el8.noarch 4/4 2020-03-19T21:16:53.010 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-mgr-cephadm-2:15.1.1-234.g787db24.el8.noarch 4/4 2020-03-19T21:16:53.010 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mgr-cephadm-2:15.1.1-234.g787db24.el8.noarch 1/4 2020-03-19T21:16:53.010 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-execnet-1.7.1-1.el8.2.noarch 2/4 2020-03-19T21:16:53.010 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-remoto-1.1.4-4.el8.noarch 3/4 2020-03-19T21:16:54.949 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-apipkg-1.5-6.el8.noarch 4/4 2020-03-19T21:16:54.949 INFO:teuthology.orchestra.run.smithi170.stdout:Installed products updated. 2020-03-19T21:16:55.152 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:16:55.152 INFO:teuthology.orchestra.run.smithi170.stdout:Installed: 2020-03-19T21:16:55.153 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-cephadm-2:15.1.1-234.g787db24.el8.noarch 2020-03-19T21:16:55.153 INFO:teuthology.orchestra.run.smithi170.stdout: python3-execnet-1.7.1-1.el8.2.noarch 2020-03-19T21:16:55.153 INFO:teuthology.orchestra.run.smithi170.stdout: python3-remoto-1.1.4-4.el8.noarch 2020-03-19T21:16:55.153 INFO:teuthology.orchestra.run.smithi170.stdout: python3-apipkg-1.5-6.el8.noarch 2020-03-19T21:16:55.154 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:16:55.154 INFO:teuthology.orchestra.run.smithi170.stdout:Complete! 2020-03-19T21:16:55.298 INFO:teuthology.orchestra.run.smithi170:> sudo yum -y install ceph-fuse 2020-03-19T21:16:55.652 INFO:teuthology.orchestra.run.smithi170.stdout:Updating Subscription Management repositories. 2020-03-19T21:16:56.228 INFO:teuthology.orchestra.run.smithi170.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 79 kB/s | 2.4 kB 00:00 2020-03-19T21:16:56.302 INFO:teuthology.orchestra.run.smithi170.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 90 kB/s | 2.8 kB 00:00 2020-03-19T21:16:57.067 INFO:teuthology.orchestra.run.smithi170.stdout:Dependencies resolved. 2020-03-19T21:16:57.067 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2020-03-19T21:16:57.068 INFO:teuthology.orchestra.run.smithi170.stdout: Package Arch Version Repository Size 2020-03-19T21:16:57.068 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2020-03-19T21:16:57.068 INFO:teuthology.orchestra.run.smithi170.stdout:Installing: 2020-03-19T21:16:57.068 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-fuse x86_64 2:15.1.1-234.g787db24.el8 ceph 572 k 2020-03-19T21:16:57.068 INFO:teuthology.orchestra.run.smithi170.stdout:Installing dependencies: 2020-03-19T21:16:57.068 INFO:teuthology.orchestra.run.smithi170.stdout: fuse x86_64 2.9.7-12.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2020-03-19T21:16:57.069 INFO:teuthology.orchestra.run.smithi170.stdout: fuse-common 2020-03-19T21:16:57.069 INFO:teuthology.orchestra.run.smithi170.stdout: x86_64 3.2.1-12.el8 rhel-8-for-x86_64-baseos-rpms 21 k 2020-03-19T21:16:57.069 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:16:57.069 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction Summary 2020-03-19T21:16:57.069 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2020-03-19T21:16:57.069 INFO:teuthology.orchestra.run.smithi170.stdout:Install 3 Packages 2020-03-19T21:16:57.069 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:16:57.070 INFO:teuthology.orchestra.run.smithi170.stdout:Total download size: 676 k 2020-03-19T21:16:57.070 INFO:teuthology.orchestra.run.smithi170.stdout:Installed size: 2.1 M 2020-03-19T21:16:57.070 INFO:teuthology.orchestra.run.smithi170.stdout:Downloading Packages: 2020-03-19T21:16:57.111 INFO:teuthology.orchestra.run.smithi170.stdout:(1/3): fuse-2.9.7-12.el8.x86_64.rpm 1.9 MB/s | 83 kB 00:00 2020-03-19T21:16:57.113 INFO:teuthology.orchestra.run.smithi170.stdout:(2/3): fuse-common-3.2.1-12.el8.x86_64.rpm 486 kB/s | 21 kB 00:00 2020-03-19T21:16:57.362 INFO:teuthology.orchestra.run.smithi170.stdout:(3/3): ceph-fuse-15.1.1-234.g787db24.el8.x86_64 1.9 MB/s | 572 kB 00:00 2020-03-19T21:16:57.362 INFO:teuthology.orchestra.run.smithi170.stdout:-------------------------------------------------------------------------------- 2020-03-19T21:16:57.363 INFO:teuthology.orchestra.run.smithi170.stdout:Total 2.2 MB/s | 676 kB 00:00 2020-03-19T21:16:57.367 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction check 2020-03-19T21:16:57.388 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction check succeeded. 2020-03-19T21:16:57.388 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction test 2020-03-19T21:16:57.455 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction test succeeded. 2020-03-19T21:16:57.455 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction 2020-03-19T21:16:57.620 INFO:teuthology.orchestra.run.smithi170.stdout: Preparing : 1/1 2020-03-19T21:16:57.620 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : fuse-common-3.2.1-12.el8.x86_64 1/3 2020-03-19T21:16:57.621 INFO:teuthology.orchestra.run.smithi170.stdout:warning: /etc/fuse.conf created as /etc/fuse.conf.rpmnew 2020-03-19T21:16:57.621 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:16:58.064 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : fuse-2.9.7-12.el8.x86_64 2/3 2020-03-19T21:16:58.264 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-fuse-2:15.1.1-234.g787db24.el8.x86_64 3/3 2020-03-19T21:16:58.581 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-fuse-2:15.1.1-234.g787db24.el8.x86_64 3/3 2020-03-19T21:16:58.581 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-fuse-2:15.1.1-234.g787db24.el8.x86_64 1/3 2020-03-19T21:16:58.581 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : fuse-2.9.7-12.el8.x86_64 2/3 2020-03-19T21:17:00.759 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : fuse-common-3.2.1-12.el8.x86_64 3/3 2020-03-19T21:17:00.759 INFO:teuthology.orchestra.run.smithi170.stdout:Installed products updated. 2020-03-19T21:17:00.960 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:17:00.961 INFO:teuthology.orchestra.run.smithi170.stdout:Installed: 2020-03-19T21:17:00.961 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-fuse-2:15.1.1-234.g787db24.el8.x86_64 fuse-2.9.7-12.el8.x86_64 2020-03-19T21:17:00.961 INFO:teuthology.orchestra.run.smithi170.stdout: fuse-common-3.2.1-12.el8.x86_64 2020-03-19T21:17:00.961 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:17:00.961 INFO:teuthology.orchestra.run.smithi170.stdout:Complete! 2020-03-19T21:17:01.130 INFO:teuthology.orchestra.run.smithi170:> sudo yum -y install librados-devel 2020-03-19T21:17:01.484 INFO:teuthology.orchestra.run.smithi170.stdout:Updating Subscription Management repositories. 2020-03-19T21:17:02.186 INFO:teuthology.orchestra.run.smithi170.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 74 kB/s | 2.4 kB 00:00 2020-03-19T21:17:02.256 INFO:teuthology.orchestra.run.smithi170.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 91 kB/s | 2.8 kB 00:00 2020-03-19T21:17:03.013 INFO:teuthology.orchestra.run.smithi170.stdout:Dependencies resolved. 2020-03-19T21:17:03.013 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2020-03-19T21:17:03.014 INFO:teuthology.orchestra.run.smithi170.stdout: Package Arch Version Repo Size 2020-03-19T21:17:03.014 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2020-03-19T21:17:03.014 INFO:teuthology.orchestra.run.smithi170.stdout:Installing: 2020-03-19T21:17:03.014 INFO:teuthology.orchestra.run.smithi170.stdout: librados-devel x86_64 2:15.1.1-234.g787db24.el8 ceph 141 k 2020-03-19T21:17:03.014 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:17:03.014 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction Summary 2020-03-19T21:17:03.014 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2020-03-19T21:17:03.015 INFO:teuthology.orchestra.run.smithi170.stdout:Install 1 Package 2020-03-19T21:17:03.015 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:17:03.015 INFO:teuthology.orchestra.run.smithi170.stdout:Total download size: 141 k 2020-03-19T21:17:03.015 INFO:teuthology.orchestra.run.smithi170.stdout:Installed size: 487 k 2020-03-19T21:17:03.016 INFO:teuthology.orchestra.run.smithi170.stdout:Downloading Packages: 2020-03-19T21:17:03.344 INFO:teuthology.orchestra.run.smithi170.stdout:librados-devel-15.1.1-234.g787db24.el8.x86_64.r 427 kB/s | 141 kB 00:00 2020-03-19T21:17:03.345 INFO:teuthology.orchestra.run.smithi170.stdout:-------------------------------------------------------------------------------- 2020-03-19T21:17:03.345 INFO:teuthology.orchestra.run.smithi170.stdout:Total 425 kB/s | 141 kB 00:00 2020-03-19T21:17:03.348 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction check 2020-03-19T21:17:03.354 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction check succeeded. 2020-03-19T21:17:03.354 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction test 2020-03-19T21:17:03.406 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction test succeeded. 2020-03-19T21:17:03.407 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction 2020-03-19T21:17:03.515 INFO:teuthology.orchestra.run.smithi170.stdout: Preparing : 1/1 2020-03-19T21:17:03.729 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : librados-devel-2:15.1.1-234.g787db24.el8.x86_64 1/1 2020-03-19T21:17:03.924 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: librados-devel-2:15.1.1-234.g787db24.el8.x86_64 1/1 2020-03-19T21:17:05.970 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librados-devel-2:15.1.1-234.g787db24.el8.x86_64 1/1 2020-03-19T21:17:05.970 INFO:teuthology.orchestra.run.smithi170.stdout:Installed products updated. 2020-03-19T21:17:06.557 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:17:06.557 INFO:teuthology.orchestra.run.smithi170.stdout:Installed: 2020-03-19T21:17:06.557 INFO:teuthology.orchestra.run.smithi170.stdout: librados-devel-2:15.1.1-234.g787db24.el8.x86_64 2020-03-19T21:17:06.557 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:17:06.558 INFO:teuthology.orchestra.run.smithi170.stdout:Complete! 2020-03-19T21:17:08.017 INFO:teuthology.orchestra.run.smithi170:> sudo yum -y install libcephfs2 2020-03-19T21:17:08.372 INFO:teuthology.orchestra.run.smithi170.stdout:Updating Subscription Management repositories. 2020-03-19T21:17:09.055 INFO:teuthology.orchestra.run.smithi170.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 73 kB/s | 2.4 kB 00:00 2020-03-19T21:17:09.131 INFO:teuthology.orchestra.run.smithi170.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 93 kB/s | 2.8 kB 00:00 2020-03-19T21:17:09.729 INFO:teuthology.orchestra.run.smithi170.stdout:Package libcephfs2-2:15.1.1-234.g787db24.el8.x86_64 is already installed. 2020-03-19T21:17:09.912 INFO:teuthology.orchestra.run.smithi170.stdout:Dependencies resolved. 2020-03-19T21:17:09.912 INFO:teuthology.orchestra.run.smithi170.stdout:Nothing to do. 2020-03-19T21:17:09.913 INFO:teuthology.orchestra.run.smithi170.stdout:Complete! 2020-03-19T21:17:10.084 INFO:teuthology.orchestra.run.smithi170:> sudo yum -y install libcephfs-devel 2020-03-19T21:17:10.437 INFO:teuthology.orchestra.run.smithi170.stdout:Updating Subscription Management repositories. 2020-03-19T21:17:11.111 INFO:teuthology.orchestra.run.smithi170.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 79 kB/s | 2.4 kB 00:00 2020-03-19T21:17:11.180 INFO:teuthology.orchestra.run.smithi170.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 96 kB/s | 2.8 kB 00:00 2020-03-19T21:17:11.928 INFO:teuthology.orchestra.run.smithi170.stdout:Dependencies resolved. 2020-03-19T21:17:11.929 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2020-03-19T21:17:11.929 INFO:teuthology.orchestra.run.smithi170.stdout: Package Arch Version Repo Size 2020-03-19T21:17:11.929 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2020-03-19T21:17:11.929 INFO:teuthology.orchestra.run.smithi170.stdout:Installing: 2020-03-19T21:17:11.929 INFO:teuthology.orchestra.run.smithi170.stdout: libcephfs-devel x86_64 2:15.1.1-234.g787db24.el8 ceph 21 k 2020-03-19T21:17:11.930 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:17:11.930 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction Summary 2020-03-19T21:17:11.930 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2020-03-19T21:17:11.930 INFO:teuthology.orchestra.run.smithi170.stdout:Install 1 Package 2020-03-19T21:17:11.930 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:17:11.931 INFO:teuthology.orchestra.run.smithi170.stdout:Total download size: 21 k 2020-03-19T21:17:11.931 INFO:teuthology.orchestra.run.smithi170.stdout:Installed size: 76 k 2020-03-19T21:17:11.931 INFO:teuthology.orchestra.run.smithi170.stdout:Downloading Packages: 2020-03-19T21:17:12.095 INFO:teuthology.orchestra.run.smithi170.stdout:libcephfs-devel-15.1.1-234.g787db24.el8.x86_64. 129 kB/s | 21 kB 00:00 2020-03-19T21:17:12.095 INFO:teuthology.orchestra.run.smithi170.stdout:-------------------------------------------------------------------------------- 2020-03-19T21:17:12.096 INFO:teuthology.orchestra.run.smithi170.stdout:Total 128 kB/s | 21 kB 00:00 2020-03-19T21:17:12.098 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction check 2020-03-19T21:17:12.101 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction check succeeded. 2020-03-19T21:17:12.104 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction test 2020-03-19T21:17:12.105 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction test succeeded. 2020-03-19T21:17:12.105 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction 2020-03-19T21:17:12.153 INFO:teuthology.orchestra.run.smithi170.stdout: Preparing : 1/1 2020-03-19T21:17:12.336 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libcephfs-devel-2:15.1.1-234.g787db24.el8.x86_64 1/1 2020-03-19T21:17:12.528 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: libcephfs-devel-2:15.1.1-234.g787db24.el8.x86_64 1/1 2020-03-19T21:17:12.862 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libcephfs-devel-2:15.1.1-234.g787db24.el8.x86_64 1/1 2020-03-19T21:17:12.863 INFO:teuthology.orchestra.run.smithi170.stdout:Last metadata expiration check: 0:00:01 ago on Thu 19 Mar 2020 09:17:11 PM UTC. 2020-03-19T21:17:14.529 INFO:teuthology.orchestra.run.smithi170.stdout:Installed products updated. 2020-03-19T21:17:14.743 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:17:14.743 INFO:teuthology.orchestra.run.smithi170.stdout:Installed: 2020-03-19T21:17:14.743 INFO:teuthology.orchestra.run.smithi170.stdout: libcephfs-devel-2:15.1.1-234.g787db24.el8.x86_64 2020-03-19T21:17:14.743 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:17:14.744 INFO:teuthology.orchestra.run.smithi170.stdout:Complete! 2020-03-19T21:17:14.895 INFO:teuthology.orchestra.run.smithi170:> sudo yum -y install librados2 2020-03-19T21:17:15.246 INFO:teuthology.orchestra.run.smithi170.stdout:Updating Subscription Management repositories. 2020-03-19T21:17:15.838 INFO:teuthology.orchestra.run.smithi170.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 68 kB/s | 2.4 kB 00:00 2020-03-19T21:17:15.909 INFO:teuthology.orchestra.run.smithi170.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 70 kB/s | 2.8 kB 00:00 2020-03-19T21:17:16.511 INFO:teuthology.orchestra.run.smithi170.stdout:Package librados2-2:15.1.1-234.g787db24.el8.x86_64 is already installed. 2020-03-19T21:17:16.692 INFO:teuthology.orchestra.run.smithi170.stdout:Dependencies resolved. 2020-03-19T21:17:16.693 INFO:teuthology.orchestra.run.smithi170.stdout:Nothing to do. 2020-03-19T21:17:16.694 INFO:teuthology.orchestra.run.smithi170.stdout:Complete! 2020-03-19T21:17:16.875 INFO:teuthology.orchestra.run.smithi170:> sudo yum -y install librbd1 2020-03-19T21:17:17.230 INFO:teuthology.orchestra.run.smithi170.stdout:Updating Subscription Management repositories. 2020-03-19T21:17:18.736 INFO:teuthology.orchestra.run.smithi170.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 77 kB/s | 2.4 kB 00:00 2020-03-19T21:17:18.807 INFO:teuthology.orchestra.run.smithi170.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 92 kB/s | 2.8 kB 00:00 2020-03-19T21:17:19.394 INFO:teuthology.orchestra.run.smithi170.stdout:Package librbd1-2:15.1.1-234.g787db24.el8.x86_64 is already installed. 2020-03-19T21:17:19.576 INFO:teuthology.orchestra.run.smithi170.stdout:Dependencies resolved. 2020-03-19T21:17:19.577 INFO:teuthology.orchestra.run.smithi170.stdout:Nothing to do. 2020-03-19T21:17:19.577 INFO:teuthology.orchestra.run.smithi170.stdout:Complete! 2020-03-19T21:17:19.710 INFO:teuthology.orchestra.run.smithi170:> sudo yum -y install python3-rados 2020-03-19T21:17:20.062 INFO:teuthology.orchestra.run.smithi170.stdout:Updating Subscription Management repositories. 2020-03-19T21:17:20.727 INFO:teuthology.orchestra.run.smithi170.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 74 kB/s | 2.4 kB 00:00 2020-03-19T21:17:20.798 INFO:teuthology.orchestra.run.smithi170.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 92 kB/s | 2.8 kB 00:00 2020-03-19T21:17:21.368 INFO:teuthology.orchestra.run.smithi170.stdout:Package python3-rados-2:15.1.1-234.g787db24.el8.x86_64 is already installed. 2020-03-19T21:17:21.550 INFO:teuthology.orchestra.run.smithi170.stdout:Dependencies resolved. 2020-03-19T21:17:21.551 INFO:teuthology.orchestra.run.smithi170.stdout:Nothing to do. 2020-03-19T21:17:21.551 INFO:teuthology.orchestra.run.smithi170.stdout:Complete! 2020-03-19T21:17:21.709 INFO:teuthology.orchestra.run.smithi170:> sudo yum -y install python3-rgw 2020-03-19T21:17:22.061 INFO:teuthology.orchestra.run.smithi170.stdout:Updating Subscription Management repositories. 2020-03-19T21:17:22.702 INFO:teuthology.orchestra.run.smithi170.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 63 kB/s | 2.4 kB 00:00 2020-03-19T21:17:22.775 INFO:teuthology.orchestra.run.smithi170.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 90 kB/s | 2.8 kB 00:00 2020-03-19T21:17:23.343 INFO:teuthology.orchestra.run.smithi170.stdout:Package python3-rgw-2:15.1.1-234.g787db24.el8.x86_64 is already installed. 2020-03-19T21:17:23.524 INFO:teuthology.orchestra.run.smithi170.stdout:Dependencies resolved. 2020-03-19T21:17:23.525 INFO:teuthology.orchestra.run.smithi170.stdout:Nothing to do. 2020-03-19T21:17:23.525 INFO:teuthology.orchestra.run.smithi170.stdout:Complete! 2020-03-19T21:17:23.687 INFO:teuthology.orchestra.run.smithi170:> sudo yum -y install python3-cephfs 2020-03-19T21:17:24.037 INFO:teuthology.orchestra.run.smithi170.stdout:Updating Subscription Management repositories. 2020-03-19T21:17:24.882 INFO:teuthology.orchestra.run.smithi170.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 77 kB/s | 2.4 kB 00:00 2020-03-19T21:17:24.956 INFO:teuthology.orchestra.run.smithi170.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 86 kB/s | 2.8 kB 00:00 2020-03-19T21:17:25.559 INFO:teuthology.orchestra.run.smithi170.stdout:Package python3-cephfs-2:15.1.1-234.g787db24.el8.x86_64 is already installed. 2020-03-19T21:17:25.743 INFO:teuthology.orchestra.run.smithi170.stdout:Dependencies resolved. 2020-03-19T21:17:25.744 INFO:teuthology.orchestra.run.smithi170.stdout:Nothing to do. 2020-03-19T21:17:25.744 INFO:teuthology.orchestra.run.smithi170.stdout:Complete! 2020-03-19T21:17:25.904 INFO:teuthology.orchestra.run.smithi170:> sudo yum -y install python3-rbd 2020-03-19T21:17:26.256 INFO:teuthology.orchestra.run.smithi170.stdout:Updating Subscription Management repositories. 2020-03-19T21:17:27.083 INFO:teuthology.orchestra.run.smithi170.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 75 kB/s | 2.4 kB 00:00 2020-03-19T21:17:27.168 INFO:teuthology.orchestra.run.smithi170.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 66 kB/s | 2.8 kB 00:00 2020-03-19T21:17:27.767 INFO:teuthology.orchestra.run.smithi170.stdout:Package python3-rbd-2:15.1.1-234.g787db24.el8.x86_64 is already installed. 2020-03-19T21:17:27.950 INFO:teuthology.orchestra.run.smithi170.stdout:Dependencies resolved. 2020-03-19T21:17:27.950 INFO:teuthology.orchestra.run.smithi170.stdout:Nothing to do. 2020-03-19T21:17:27.951 INFO:teuthology.orchestra.run.smithi170.stdout:Complete! 2020-03-19T21:17:28.130 INFO:teuthology.orchestra.run.smithi170:> sudo yum -y install rbd-fuse 2020-03-19T21:17:28.483 INFO:teuthology.orchestra.run.smithi170.stdout:Updating Subscription Management repositories. 2020-03-19T21:17:29.260 INFO:teuthology.orchestra.run.smithi170.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 40 kB/s | 2.4 kB 00:00 2020-03-19T21:17:29.338 INFO:teuthology.orchestra.run.smithi170.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 78 kB/s | 2.8 kB 00:00 2020-03-19T21:17:30.075 INFO:teuthology.orchestra.run.smithi170.stdout:Dependencies resolved. 2020-03-19T21:17:30.075 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2020-03-19T21:17:30.075 INFO:teuthology.orchestra.run.smithi170.stdout: Package Architecture Version Repository Size 2020-03-19T21:17:30.075 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2020-03-19T21:17:30.076 INFO:teuthology.orchestra.run.smithi170.stdout:Installing: 2020-03-19T21:17:30.076 INFO:teuthology.orchestra.run.smithi170.stdout: rbd-fuse x86_64 2:15.1.1-234.g787db24.el8 ceph 72 k 2020-03-19T21:17:30.076 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:17:30.076 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction Summary 2020-03-19T21:17:30.076 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2020-03-19T21:17:30.076 INFO:teuthology.orchestra.run.smithi170.stdout:Install 1 Package 2020-03-19T21:17:30.076 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:17:30.077 INFO:teuthology.orchestra.run.smithi170.stdout:Total download size: 72 k 2020-03-19T21:17:30.077 INFO:teuthology.orchestra.run.smithi170.stdout:Installed size: 230 k 2020-03-19T21:17:30.077 INFO:teuthology.orchestra.run.smithi170.stdout:Downloading Packages: 2020-03-19T21:17:30.271 INFO:teuthology.orchestra.run.smithi170.stdout:rbd-fuse-15.1.1-234.g787db24.el8.x86_64.rpm 371 kB/s | 72 kB 00:00 2020-03-19T21:17:30.272 INFO:teuthology.orchestra.run.smithi170.stdout:-------------------------------------------------------------------------------- 2020-03-19T21:17:30.272 INFO:teuthology.orchestra.run.smithi170.stdout:Total 369 kB/s | 72 kB 00:00 2020-03-19T21:17:30.274 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction check 2020-03-19T21:17:30.285 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction check succeeded. 2020-03-19T21:17:30.285 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction test 2020-03-19T21:17:30.336 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction test succeeded. 2020-03-19T21:17:30.337 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction 2020-03-19T21:17:30.436 INFO:teuthology.orchestra.run.smithi170.stdout: Preparing : 1/1 2020-03-19T21:17:30.641 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : rbd-fuse-2:15.1.1-234.g787db24.el8.x86_64 1/1 2020-03-19T21:17:30.858 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: rbd-fuse-2:15.1.1-234.g787db24.el8.x86_64 1/1 2020-03-19T21:17:32.980 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : rbd-fuse-2:15.1.1-234.g787db24.el8.x86_64 1/1 2020-03-19T21:17:32.980 INFO:teuthology.orchestra.run.smithi170.stdout:Installed products updated. 2020-03-19T21:17:33.179 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:17:33.180 INFO:teuthology.orchestra.run.smithi170.stdout:Installed: 2020-03-19T21:17:33.180 INFO:teuthology.orchestra.run.smithi170.stdout: rbd-fuse-2:15.1.1-234.g787db24.el8.x86_64 2020-03-19T21:17:33.180 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:17:33.180 INFO:teuthology.orchestra.run.smithi170.stdout:Complete! 2020-03-19T21:17:33.328 INFO:teuthology.orchestra.run.smithi170:> sudo yum -y install rbd-mirror 2020-03-19T21:17:33.681 INFO:teuthology.orchestra.run.smithi170.stdout:Updating Subscription Management repositories. 2020-03-19T21:17:34.391 INFO:teuthology.orchestra.run.smithi170.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 76 kB/s | 2.4 kB 00:00 2020-03-19T21:17:34.467 INFO:teuthology.orchestra.run.smithi170.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 76 kB/s | 2.8 kB 00:00 2020-03-19T21:17:35.213 INFO:teuthology.orchestra.run.smithi170.stdout:Dependencies resolved. 2020-03-19T21:17:35.213 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2020-03-19T21:17:35.213 INFO:teuthology.orchestra.run.smithi170.stdout: Package Architecture Version Repository Size 2020-03-19T21:17:35.214 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2020-03-19T21:17:35.214 INFO:teuthology.orchestra.run.smithi170.stdout:Installing: 2020-03-19T21:17:35.214 INFO:teuthology.orchestra.run.smithi170.stdout: rbd-mirror x86_64 2:15.1.1-234.g787db24.el8 ceph 2.7 M 2020-03-19T21:17:35.214 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:17:35.214 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction Summary 2020-03-19T21:17:35.214 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2020-03-19T21:17:35.214 INFO:teuthology.orchestra.run.smithi170.stdout:Install 1 Package 2020-03-19T21:17:35.215 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:17:35.215 INFO:teuthology.orchestra.run.smithi170.stdout:Total download size: 2.7 M 2020-03-19T21:17:35.215 INFO:teuthology.orchestra.run.smithi170.stdout:Installed size: 13 M 2020-03-19T21:17:35.215 INFO:teuthology.orchestra.run.smithi170.stdout:Downloading Packages: 2020-03-19T21:17:36.865 INFO:teuthology.orchestra.run.smithi170.stdout:rbd-mirror-15.1.1-234.g787db24.el8.x86_64.rpm 1.6 MB/s | 2.7 MB 00:01 2020-03-19T21:17:36.866 INFO:teuthology.orchestra.run.smithi170.stdout:-------------------------------------------------------------------------------- 2020-03-19T21:17:36.867 INFO:teuthology.orchestra.run.smithi170.stdout:Total 1.6 MB/s | 2.7 MB 00:01 2020-03-19T21:17:36.870 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction check 2020-03-19T21:17:36.884 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction check succeeded. 2020-03-19T21:17:36.884 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction test 2020-03-19T21:17:36.948 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction test succeeded. 2020-03-19T21:17:36.949 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction 2020-03-19T21:17:37.309 INFO:teuthology.orchestra.run.smithi170.stdout: Preparing : 1/1 2020-03-19T21:17:37.493 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : rbd-mirror-2:15.1.1-234.g787db24.el8.x86_64 1/1 2020-03-19T21:17:37.847 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: rbd-mirror-2:15.1.1-234.g787db24.el8.x86_64 1/1 2020-03-19T21:17:40.129 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : rbd-mirror-2:15.1.1-234.g787db24.el8.x86_64 1/1 2020-03-19T21:17:40.130 INFO:teuthology.orchestra.run.smithi170.stdout:Installed products updated. 2020-03-19T21:17:40.337 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:17:40.337 INFO:teuthology.orchestra.run.smithi170.stdout:Installed: 2020-03-19T21:17:40.337 INFO:teuthology.orchestra.run.smithi170.stdout: rbd-mirror-2:15.1.1-234.g787db24.el8.x86_64 2020-03-19T21:17:40.337 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:17:40.337 INFO:teuthology.orchestra.run.smithi170.stdout:Complete! 2020-03-19T21:17:40.508 INFO:teuthology.orchestra.run.smithi170:> sudo yum -y install rbd-nbd 2020-03-19T21:17:40.862 INFO:teuthology.orchestra.run.smithi170.stdout:Updating Subscription Management repositories. 2020-03-19T21:17:41.556 INFO:teuthology.orchestra.run.smithi170.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 79 kB/s | 2.4 kB 00:00 2020-03-19T21:17:41.626 INFO:teuthology.orchestra.run.smithi170.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 90 kB/s | 2.8 kB 00:00 2020-03-19T21:17:42.388 INFO:teuthology.orchestra.run.smithi170.stdout:Dependencies resolved. 2020-03-19T21:17:42.388 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2020-03-19T21:17:42.389 INFO:teuthology.orchestra.run.smithi170.stdout: Package Architecture Version Repository Size 2020-03-19T21:17:42.389 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2020-03-19T21:17:42.389 INFO:teuthology.orchestra.run.smithi170.stdout:Installing: 2020-03-19T21:17:42.389 INFO:teuthology.orchestra.run.smithi170.stdout: rbd-nbd x86_64 2:15.1.1-234.g787db24.el8 ceph 141 k 2020-03-19T21:17:42.389 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:17:42.389 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction Summary 2020-03-19T21:17:42.390 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2020-03-19T21:17:42.390 INFO:teuthology.orchestra.run.smithi170.stdout:Install 1 Package 2020-03-19T21:17:42.390 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:17:42.390 INFO:teuthology.orchestra.run.smithi170.stdout:Total download size: 141 k 2020-03-19T21:17:42.390 INFO:teuthology.orchestra.run.smithi170.stdout:Installed size: 526 k 2020-03-19T21:17:42.390 INFO:teuthology.orchestra.run.smithi170.stdout:Downloading Packages: 2020-03-19T21:17:42.620 INFO:teuthology.orchestra.run.smithi170.stdout:rbd-nbd-15.1.1-234.g787db24.el8.x86_64.rpm 611 kB/s | 141 kB 00:00 2020-03-19T21:17:42.621 INFO:teuthology.orchestra.run.smithi170.stdout:-------------------------------------------------------------------------------- 2020-03-19T21:17:42.621 INFO:teuthology.orchestra.run.smithi170.stdout:Total 607 kB/s | 141 kB 00:00 2020-03-19T21:17:42.624 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction check 2020-03-19T21:17:42.634 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction check succeeded. 2020-03-19T21:17:42.634 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction test 2020-03-19T21:17:42.685 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction test succeeded. 2020-03-19T21:17:42.685 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction 2020-03-19T21:17:42.817 INFO:teuthology.orchestra.run.smithi170.stdout: Preparing : 1/1 2020-03-19T21:17:43.022 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : rbd-nbd-2:15.1.1-234.g787db24.el8.x86_64 1/1 2020-03-19T21:17:43.242 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: rbd-nbd-2:15.1.1-234.g787db24.el8.x86_64 1/1 2020-03-19T21:17:45.731 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : rbd-nbd-2:15.1.1-234.g787db24.el8.x86_64 1/1 2020-03-19T21:17:45.731 INFO:teuthology.orchestra.run.smithi170.stdout:Installed products updated. 2020-03-19T21:17:45.945 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:17:45.946 INFO:teuthology.orchestra.run.smithi170.stdout:Installed: 2020-03-19T21:17:45.946 INFO:teuthology.orchestra.run.smithi170.stdout: rbd-nbd-2:15.1.1-234.g787db24.el8.x86_64 2020-03-19T21:17:45.946 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:17:45.946 INFO:teuthology.orchestra.run.smithi170.stdout:Complete! 2020-03-19T21:17:46.093 DEBUG:teuthology.parallel:result is None 2020-03-19T21:17:46.093 INFO:teuthology.orchestra.run.smithi170:> sudo lsb_release -is 2020-03-19T21:17:46.130 INFO:teuthology.orchestra.run.smithi170.stdout:RedHatEnterprise 2020-03-19T21:17:46.131 DEBUG:teuthology.misc:System to be installed: RedHatEnterprise 2020-03-19T21:17:46.132 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=787db24c312ed9d310cd3f194b00548ede9a199c 2020-03-19T21:17:46.356 INFO:teuthology.orchestra.run.smithi170:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2020-03-19T21:17:46.375 INFO:teuthology.orchestra.run.smithi170.stdout:15.1.1-234.g787db24.el8 2020-03-19T21:17:46.376 INFO:teuthology.packaging:The installed version of ceph is 15.1.1-234.g787db24.el8 2020-03-19T21:17:46.376 INFO:teuthology.task.install:The correct ceph version 15.1.1-234.g787db24 is installed. 2020-03-19T21:17:46.377 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2020-03-19T21:17:46.378 INFO:teuthology.orchestra.run.smithi170:> sudo sh -c 'cat > /home/ubuntu/cephtest/valgrind.supp' 2020-03-19T21:17:46.445 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2020-03-19T21:17:46.445 INFO:teuthology.orchestra.run.smithi170:> sudo sh -c 'cat > /usr/bin/daemon-helper' 2020-03-19T21:17:46.512 INFO:teuthology.orchestra.run.smithi170:> sudo chmod a=rx -- /usr/bin/daemon-helper 2020-03-19T21:17:46.579 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2020-03-19T21:17:46.581 INFO:teuthology.orchestra.run.smithi170:> sudo sh -c 'cat > /usr/bin/adjust-ulimits' 2020-03-19T21:17:46.667 INFO:teuthology.orchestra.run.smithi170:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2020-03-19T21:17:46.734 INFO:teuthology.run_tasks:Running task ceph... 2020-03-19T21:17:46.801 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2020-03-19T21:17:46.801 INFO:teuthology.orchestra.run.smithi170:> sudo chmod 777 /var/log/ceph 2020-03-19T21:17:46.829 INFO:tasks.ceph:Disabling ceph logrotate... 2020-03-19T21:17:46.830 INFO:teuthology.orchestra.run.smithi170:> sudo rm -f -- /etc/logrotate.d/ceph 2020-03-19T21:17:46.895 INFO:tasks.ceph:Creating extra log directories... 2020-03-19T21:17:46.896 INFO:teuthology.orchestra.run.smithi170:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2020-03-19T21:17:46.963 INFO:tasks.ceph:Creating ceph cluster ceph... 2020-03-19T21:17:46.963 INFO:tasks.ceph:config {'mon_bind_addrvec': True, 'block_journal': None, 'fs': 'xfs', 'log_whitelist': ['overall HEALTH_', '\\(CACHE_POOL_NO_HIT_SET\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'skip_mgr_daemons': False, 'cpu_profile': set([]), 'mkfs_options': None, 'tmpfs_journal': None, 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mount_options': None, 'conf': {'global': {'debug ms': 1, 'debug mds': 20, 'osd max object namespace len': 64, 'osd max object name len': 460, 'debug client': 20, 'mon election default strategy': 3}, 'mon': {'debug mon': 20, 'debug paxos': 20, 'debug ms': 1}, 'osd': {'osd op queue cut off': 'debug_random', 'debug ms': 20, 'debug journal': 20, 'debug osd': 25, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd op queue': 'debug_random', 'debug filestore': 20}}} 2020-03-19T21:17:46.963 INFO:tasks.ceph:ctx.config {'kernel': {'sha1': 'distro', 'kdb': True}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.lock_machines': (1, 'smithi')}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'sha1': 'distro', 'kdb': True}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': {'cluster': 'ceph', 'pre-mgr-commands': ['sudo ceph config set mgr mgr/devicehealth/enable_monitoring false --force'], 'log-whitelist': ['overall HEALTH_', '\\(CACHE_POOL_NO_HIT_SET\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'conf': {'global': {'debug ms': 1, 'debug mds': 20, 'osd max object namespace len': 64, 'osd max object name len': 460, 'debug client': 20, 'mon election default strategy': 3}, 'mon': {'debug mon': 20, 'debug paxos': 20, 'debug ms': 1}, 'osd': {'osd op queue cut off': 'debug_random', 'debug ms': 20, 'debug journal': 20, 'debug osd': 25, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd op queue': 'debug_random', 'debug filestore': 20}}, 'sha1': '787db24c312ed9d310cd3f194b00548ede9a199c'}}, {'exec': {'client.0': ['ceph osd pool create data_cache 4', 'ceph osd tier add cephfs_data data_cache', 'ceph osd tier cache-mode data_cache writeback', 'ceph osd tier set-overlay cephfs_data data_cache', 'ceph osd pool set data_cache hit_set_type bloom', 'ceph osd pool set data_cache hit_set_count 8', 'ceph osd pool set data_cache hit_set_period 3600', 'ceph osd pool set data_cache min_read_recency_for_promote 0']}}, {'ceph-fuse': None}, {'exec': {'client.0': ['sudo chmod 777 $TESTDIR/mnt.0/', 'dd if=/dev/urandom of=$TESTDIR/mnt.0/foo bs=1M count=5', 'ls -al $TESTDIR/mnt.0/foo', 'truncate --size 0 $TESTDIR/mnt.0/foo', 'ls -al $TESTDIR/mnt.0/foo', 'dd if=/dev/urandom of=$TESTDIR/mnt.0/foo bs=1M count=5', 'ls -al $TESTDIR/mnt.0/foo', 'cp $TESTDIR/mnt.0/foo /tmp/foo', 'sync', 'rados -p data_cache ls -', 'sleep 10', 'rados -p data_cache ls -', 'rados -p data_cache cache-flush-evict-all', 'rados -p data_cache ls -', 'sleep 1']}}, {'exec': {'client.1': ['hexdump -C /tmp/foo | head', 'hexdump -C $TESTDIR/mnt.1/foo | head', 'cmp $TESTDIR/mnt.1/foo /tmp/foo']}}], 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_wip-sage-testing-2020-03-19-0855/qa', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/worker.smithi.16493', 'owner': 'scheduled_sage@teuthology', 'openstack': [{'volumes': {'count': 3, 'size': 10}}], 'targets': {'smithi170.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCy6Zubi7wlI+ftilAaKFs+a8x8iX8tOxNsMUGMaJfZVLRO2JNZJfD6+TZ/frqrSjdEmDDBFcR+LPdkOZ22N0T/b1DxeUxSs5KNrf3KBKUWJIQ4O4Hvm1JTtOseX52/1//e9TDD6uaGSYV8Ge0Gj6SvpUe6Tz5oMI5MlOqXQ5vP86QWpFCifq2Ms24866n8IK4aruHvrwbtfu9fQzd1gnb8eYN/JqQriLJhKnwI16rz86b9ZHx/mWD8EPB40NuLLKO7W0uXg08I8SA/uQ0kfMHSC1eecF+OqOaP700QNaCQl4IEQJRmWzrGh5W6BGoyJDk/PKmzhc+KREQwrO1qIKZjUEp3Jcu81uR6PuBnr3l+pM34e5KWOjSBt2SOkIMnA4y/2hu1WxzJfB6L+5OSYL7lsZbANz8TZ3LfsKJmeFVnZIZKncXb9o8ZIQ9t8acfJBI8uqaBV4SCszLhPFRqrBY03EDb4f/sFXjt+Y8Rxnf08XbL7pyaKTraKgW9HZ9Osh8='}, 'job_id': '4868524', 'tube': 'smithi', 'suite_branch': 'wip-sage-testing-2020-03-19-0855', 'priority': 90, 'os_version': '8.1', 'first_in_suite': False, 'branch': 'wip-sage-testing-2020-03-19-0855', 'suite': 'rados', 'email': None, 'archive_path': '/home/teuthworker/archive/sage-2020-03-19_17:25:48-rados-wip-sage-testing-2020-03-19-0855-distro-basic-smithi/4868524', 'description': 'rados/singleton-nomsgr/{all/cache-fs-trunc.yaml mon_election/connectivity.yaml rados.yaml supported-random-distro$/{rhel_8.yaml}}', 'nuke-on-error': True, 'repo': 'git://git.ceph.com/ceph-ci.git', 'last_in_suite': False, 'machine_type': 'smithi', 'teuthology_branch': 'master', 'sha1': '787db24c312ed9d310cd3f194b00548ede9a199c', 'name': 'sage-2020-03-19_17:25:48-rados-wip-sage-testing-2020-03-19-0855-distro-basic-smithi', 'roles': [['mon.a', 'mgr.x', 'mds.a', 'osd.0', 'osd.1', 'osd.2', 'client.0', 'client.1']], 'suite_relpath': 'qa', 'suite_repo': 'git://git.ceph.com/ceph-ci.git', 'suite_sha1': '787db24c312ed9d310cd3f194b00548ede9a199c', 'os_type': 'rhel', 'overrides': {'ceph': {'log-whitelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'conf': {'global': {'mon election default strategy': 3}, 'mon': {'debug mon': 20, 'debug paxos': 20, 'debug ms': 1}, 'osd': {'osd op queue cut off': 'debug_random', 'debug ms': 20, 'debug journal': 20, 'debug osd': 25, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd op queue': 'debug_random', 'debug filestore': 20}}, 'sha1': '787db24c312ed9d310cd3f194b00548ede9a199c'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}}}, 'install': {'ceph': {'sha1': '787db24c312ed9d310cd3f194b00548ede9a199c'}}, 'admin_socket': {'branch': 'wip-sage-testing-2020-03-19-0855'}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'workunit': {'sha1': '787db24c312ed9d310cd3f194b00548ede9a199c', 'branch': 'wip-sage-testing-2020-03-19-0855'}}} 2020-03-19T21:17:46.964 INFO:teuthology.orchestra.run.smithi170:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2020-03-19T21:17:47.017 INFO:teuthology.orchestra.run.smithi170:> sudo install -d -m0777 -- /var/run/ceph 2020-03-19T21:17:47.141 DEBUG:teuthology.orchestra.remote:smithi170:/scratch_devs is 72B 2020-03-19T21:17:47.190 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_4', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_1'] 2020-03-19T21:17:47.190 INFO:teuthology.orchestra.run.smithi170:> stat /dev/vg_nvme/lv_4 && sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_4 2020-03-19T21:17:47.204 INFO:teuthology.orchestra.run.smithi170.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-1 2020-03-19T21:17:47.205 INFO:teuthology.orchestra.run.smithi170.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-03-19T21:17:47.205 INFO:teuthology.orchestra.run.smithi170.stdout:Device: 6h/6d Inode: 64668 Links: 1 2020-03-19T21:17:47.205 INFO:teuthology.orchestra.run.smithi170.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-03-19T21:17:47.205 INFO:teuthology.orchestra.run.smithi170.stdout:Context: system_u:object_r:device_t:s0 2020-03-19T21:17:47.205 INFO:teuthology.orchestra.run.smithi170.stdout:Access: 2020-03-19 21:17:37.677536228 +0000 2020-03-19T21:17:47.205 INFO:teuthology.orchestra.run.smithi170.stdout:Modify: 2020-03-19 21:14:21.715859681 +0000 2020-03-19T21:17:47.206 INFO:teuthology.orchestra.run.smithi170.stdout:Change: 2020-03-19 21:14:21.715859681 +0000 2020-03-19T21:17:47.206 INFO:teuthology.orchestra.run.smithi170.stdout: Birth: - 2020-03-19T21:17:47.217 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records in 2020-03-19T21:17:47.217 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records out 2020-03-19T21:17:47.218 INFO:teuthology.orchestra.run.smithi170.stderr:512 bytes copied, 0.000258222 s, 2.0 MB/s 2020-03-19T21:17:47.220 INFO:teuthology.orchestra.run.smithi170:> stat /dev/vg_nvme/lv_3 && sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_3 2020-03-19T21:17:47.274 INFO:teuthology.orchestra.run.smithi170.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2020-03-19T21:17:47.274 INFO:teuthology.orchestra.run.smithi170.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-03-19T21:17:47.274 INFO:teuthology.orchestra.run.smithi170.stdout:Device: 6h/6d Inode: 63132 Links: 1 2020-03-19T21:17:47.274 INFO:teuthology.orchestra.run.smithi170.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-03-19T21:17:47.274 INFO:teuthology.orchestra.run.smithi170.stdout:Context: system_u:object_r:device_t:s0 2020-03-19T21:17:47.274 INFO:teuthology.orchestra.run.smithi170.stdout:Access: 2020-03-19 21:17:37.677536228 +0000 2020-03-19T21:17:47.275 INFO:teuthology.orchestra.run.smithi170.stdout:Modify: 2020-03-19 21:14:22.050851302 +0000 2020-03-19T21:17:47.275 INFO:teuthology.orchestra.run.smithi170.stdout:Change: 2020-03-19 21:14:22.050851302 +0000 2020-03-19T21:17:47.275 INFO:teuthology.orchestra.run.smithi170.stdout: Birth: - 2020-03-19T21:17:47.286 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records in 2020-03-19T21:17:47.287 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records out 2020-03-19T21:17:47.287 INFO:teuthology.orchestra.run.smithi170.stderr:512 bytes copied, 0.000160004 s, 3.2 MB/s 2020-03-19T21:17:47.289 INFO:teuthology.orchestra.run.smithi170:> stat /dev/vg_nvme/lv_2 && sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_2 2020-03-19T21:17:47.343 INFO:teuthology.orchestra.run.smithi170.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-3 2020-03-19T21:17:47.343 INFO:teuthology.orchestra.run.smithi170.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-03-19T21:17:47.343 INFO:teuthology.orchestra.run.smithi170.stdout:Device: 6h/6d Inode: 61386 Links: 1 2020-03-19T21:17:47.343 INFO:teuthology.orchestra.run.smithi170.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-03-19T21:17:47.343 INFO:teuthology.orchestra.run.smithi170.stdout:Context: system_u:object_r:device_t:s0 2020-03-19T21:17:47.344 INFO:teuthology.orchestra.run.smithi170.stdout:Access: 2020-03-19 21:17:37.677536228 +0000 2020-03-19T21:17:47.344 INFO:teuthology.orchestra.run.smithi170.stdout:Modify: 2020-03-19 21:14:22.388842846 +0000 2020-03-19T21:17:47.344 INFO:teuthology.orchestra.run.smithi170.stdout:Change: 2020-03-19 21:14:22.388842846 +0000 2020-03-19T21:17:47.344 INFO:teuthology.orchestra.run.smithi170.stdout: Birth: - 2020-03-19T21:17:47.356 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records in 2020-03-19T21:17:47.356 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records out 2020-03-19T21:17:47.356 INFO:teuthology.orchestra.run.smithi170.stderr:512 bytes copied, 0.000180861 s, 2.8 MB/s 2020-03-19T21:17:47.358 INFO:teuthology.orchestra.run.smithi170:> stat /dev/vg_nvme/lv_1 && sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_1 2020-03-19T21:17:47.412 INFO:teuthology.orchestra.run.smithi170.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-4 2020-03-19T21:17:47.413 INFO:teuthology.orchestra.run.smithi170.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-03-19T21:17:47.413 INFO:teuthology.orchestra.run.smithi170.stdout:Device: 6h/6d Inode: 66580 Links: 1 2020-03-19T21:17:47.413 INFO:teuthology.orchestra.run.smithi170.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-03-19T21:17:47.413 INFO:teuthology.orchestra.run.smithi170.stdout:Context: system_u:object_r:device_t:s0 2020-03-19T21:17:47.413 INFO:teuthology.orchestra.run.smithi170.stdout:Access: 2020-03-19 21:17:37.677536228 +0000 2020-03-19T21:17:47.413 INFO:teuthology.orchestra.run.smithi170.stdout:Modify: 2020-03-19 21:14:22.754833690 +0000 2020-03-19T21:17:47.413 INFO:teuthology.orchestra.run.smithi170.stdout:Change: 2020-03-19 21:14:22.754833690 +0000 2020-03-19T21:17:47.414 INFO:teuthology.orchestra.run.smithi170.stdout: Birth: - 2020-03-19T21:17:47.425 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records in 2020-03-19T21:17:47.426 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records out 2020-03-19T21:17:47.426 INFO:teuthology.orchestra.run.smithi170.stderr:512 bytes copied, 0.000236034 s, 2.2 MB/s 2020-03-19T21:17:47.428 INFO:tasks.ceph:fs option selected, checking for scratch devs 2020-03-19T21:17:47.429 INFO:tasks.ceph:found devs: ['/dev/vg_nvme/lv_4', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_1'] 2020-03-19T21:17:47.429 WARNING:teuthology.misc:Entering get_wwn_id_map, a deprecated function that will be removed 2020-03-19T21:17:47.429 INFO:tasks.ceph:dev map: {'osd.1': '/dev/vg_nvme/lv_1', 'osd.0': '/dev/vg_nvme/lv_4', 'osd.2': '/dev/vg_nvme/lv_2'} 2020-03-19T21:17:47.429 INFO:tasks.ceph:Generating config... 2020-03-19T21:17:47.431 INFO:tasks.ceph:[global] debug ms = 1 2020-03-19T21:17:47.431 INFO:tasks.ceph:[global] debug mds = 20 2020-03-19T21:17:47.432 INFO:tasks.ceph:[global] osd max object namespace len = 64 2020-03-19T21:17:47.432 INFO:tasks.ceph:[global] osd max object name len = 460 2020-03-19T21:17:47.432 INFO:tasks.ceph:[global] debug client = 20 2020-03-19T21:17:47.432 INFO:tasks.ceph:[global] mon election default strategy = 3 2020-03-19T21:17:47.432 INFO:tasks.ceph:[mon] debug mon = 20 2020-03-19T21:17:47.432 INFO:tasks.ceph:[mon] debug paxos = 20 2020-03-19T21:17:47.432 INFO:tasks.ceph:[mon] debug ms = 1 2020-03-19T21:17:47.433 INFO:tasks.ceph:[osd] osd op queue cut off = debug_random 2020-03-19T21:17:47.433 INFO:tasks.ceph:[osd] debug ms = 20 2020-03-19T21:17:47.433 INFO:tasks.ceph:[osd] debug journal = 20 2020-03-19T21:17:47.433 INFO:tasks.ceph:[osd] debug osd = 25 2020-03-19T21:17:47.433 INFO:tasks.ceph:[osd] osd debug verify cached snaps = True 2020-03-19T21:17:47.433 INFO:tasks.ceph:[osd] osd debug verify missing on start = True 2020-03-19T21:17:47.433 INFO:tasks.ceph:[osd] osd op queue = debug_random 2020-03-19T21:17:47.433 INFO:tasks.ceph:[osd] debug filestore = 20 2020-03-19T21:17:47.434 INFO:tasks.ceph:Setting up mon.a... 2020-03-19T21:17:47.434 INFO:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2020-03-19T21:17:47.517 INFO:teuthology.orchestra.run.smithi170.stdout:creating /etc/ceph/ceph.keyring 2020-03-19T21:17:47.519 INFO:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2020-03-19T21:17:47.560 INFO:teuthology.orchestra.run.smithi170:> sudo chmod 0644 /etc/ceph/ceph.keyring 2020-03-19T21:17:47.627 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.170')] 2020-03-19T21:17:47.627 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon host': '172.21.15.170', 'debug ms': 1, 'debug mds': 20, 'osd max object namespace len': 64, 'osd max object name len': 460, 'debug client': 20, 'mon election default strategy': 3}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'osd op queue cut off': 'debug_random', 'debug ms': 20, 'debug journal': 20, 'debug osd': 25, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd op queue': 'debug_random', 'debug filestore': 20}, 'mgr': {'debug ms': '1', 'debug mgr': '20', 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10'}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok'}, 'mon.a': {}, 'mds.a': {}} 2020-03-19T21:17:47.628 INFO:teuthology.orchestra.run.smithi170:> cat > /home/ubuntu/cephtest/ceph.tmp.conf 2020-03-19T21:17:47.681 INFO:teuthology.orchestra.run.smithi170:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.170 --print /home/ubuntu/cephtest/ceph.monmap 2020-03-19T21:17:47.754 INFO:teuthology.orchestra.run.smithi170.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2020-03-19T21:17:47.754 INFO:teuthology.orchestra.run.smithi170.stdout:monmaptool: generated fsid 12d0a33b-6b40-4398-90b4-6b5df1da388e 2020-03-19T21:17:47.754 INFO:teuthology.orchestra.run.smithi170.stdout:epoch 0 2020-03-19T21:17:47.754 INFO:teuthology.orchestra.run.smithi170.stdout:fsid 12d0a33b-6b40-4398-90b4-6b5df1da388e 2020-03-19T21:17:47.754 INFO:teuthology.orchestra.run.smithi170.stdout:last_changed 2020-03-19T21:17:47.753317+0000 2020-03-19T21:17:47.754 INFO:teuthology.orchestra.run.smithi170.stdout:created 2020-03-19T21:17:47.753317+0000 2020-03-19T21:17:47.755 INFO:teuthology.orchestra.run.smithi170.stdout:min_mon_release 0 (unknown) 2020-03-19T21:17:47.755 INFO:teuthology.orchestra.run.smithi170.stdout:election_strategy: 3 2020-03-19T21:17:47.755 INFO:teuthology.orchestra.run.smithi170.stdout:0: [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] mon.a 2020-03-19T21:17:47.755 INFO:teuthology.orchestra.run.smithi170.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (1 monitors) 2020-03-19T21:17:47.756 INFO:teuthology.orchestra.run.smithi170:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2020-03-19T21:17:47.810 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 12d0a33b-6b40-4398-90b4-6b5df1da388e... 2020-03-19T21:17:47.811 INFO:teuthology.orchestra.run.smithi170:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2020-03-19T21:17:47.902 INFO:teuthology.orchestra.run.smithi170.stdout:[global] 2020-03-19T21:17:47.902 INFO:teuthology.orchestra.run.smithi170.stdout: chdir = "" 2020-03-19T21:17:47.902 INFO:teuthology.orchestra.run.smithi170.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2020-03-19T21:17:47.903 INFO:teuthology.orchestra.run.smithi170.stdout: auth supported = cephx 2020-03-19T21:17:47.903 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:17:47.903 INFO:teuthology.orchestra.run.smithi170.stdout: filestore xattr use omap = true 2020-03-19T21:17:47.903 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:17:47.903 INFO:teuthology.orchestra.run.smithi170.stdout: mon clock drift allowed = 1.000 2020-03-19T21:17:47.903 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:17:47.903 INFO:teuthology.orchestra.run.smithi170.stdout: osd crush chooseleaf type = 0 2020-03-19T21:17:47.903 INFO:teuthology.orchestra.run.smithi170.stdout: auth debug = true 2020-03-19T21:17:47.904 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:17:47.904 INFO:teuthology.orchestra.run.smithi170.stdout: ms die on old message = true 2020-03-19T21:17:47.904 INFO:teuthology.orchestra.run.smithi170.stdout: ms die on bug = true 2020-03-19T21:17:47.904 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:17:47.904 INFO:teuthology.orchestra.run.smithi170.stdout: mon max pg per osd = 10000 # >= luminous 2020-03-19T21:17:47.904 INFO:teuthology.orchestra.run.smithi170.stdout: mon pg warn max object skew = 0 2020-03-19T21:17:47.904 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:17:47.905 INFO:teuthology.orchestra.run.smithi170.stdout: # disable pg_autoscaler by default for new pools 2020-03-19T21:17:47.905 INFO:teuthology.orchestra.run.smithi170.stdout: osd_pool_default_pg_autoscale_mode = off 2020-03-19T21:17:47.905 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:17:47.905 INFO:teuthology.orchestra.run.smithi170.stdout: osd pool default size = 2 2020-03-19T21:17:47.905 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:17:47.905 INFO:teuthology.orchestra.run.smithi170.stdout: mon osd allow primary affinity = true 2020-03-19T21:17:47.905 INFO:teuthology.orchestra.run.smithi170.stdout: mon osd allow pg remap = true 2020-03-19T21:17:47.906 INFO:teuthology.orchestra.run.smithi170.stdout: mon warn on legacy crush tunables = false 2020-03-19T21:17:47.906 INFO:teuthology.orchestra.run.smithi170.stdout: mon warn on crush straw calc version zero = false 2020-03-19T21:17:47.906 INFO:teuthology.orchestra.run.smithi170.stdout: mon warn on no sortbitwise = false 2020-03-19T21:17:47.906 INFO:teuthology.orchestra.run.smithi170.stdout: mon warn on osd down out interval zero = false 2020-03-19T21:17:47.906 INFO:teuthology.orchestra.run.smithi170.stdout: mon warn on too few osds = false 2020-03-19T21:17:47.906 INFO:teuthology.orchestra.run.smithi170.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2020-03-19T21:17:47.906 INFO:teuthology.orchestra.run.smithi170.stdout: mon_warn_on_pool_no_redundancy = false 2020-03-19T21:17:47.907 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:17:47.907 INFO:teuthology.orchestra.run.smithi170.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2020-03-19T21:17:47.907 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:17:47.907 INFO:teuthology.orchestra.run.smithi170.stdout: osd default data pool replay window = 5 2020-03-19T21:17:47.907 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:17:47.907 INFO:teuthology.orchestra.run.smithi170.stdout: mon allow pool delete = true 2020-03-19T21:17:47.907 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:17:47.908 INFO:teuthology.orchestra.run.smithi170.stdout: mon cluster log file level = debug 2020-03-19T21:17:47.908 INFO:teuthology.orchestra.run.smithi170.stdout: debug asserts on shutdown = true 2020-03-19T21:17:47.908 INFO:teuthology.orchestra.run.smithi170.stdout: mon host = 172.21.15.170 2020-03-19T21:17:47.908 INFO:teuthology.orchestra.run.smithi170.stdout: debug ms = 1 2020-03-19T21:17:47.908 INFO:teuthology.orchestra.run.smithi170.stdout: debug mds = 20 2020-03-19T21:17:47.908 INFO:teuthology.orchestra.run.smithi170.stdout: osd max object namespace len = 64 2020-03-19T21:17:47.908 INFO:teuthology.orchestra.run.smithi170.stdout: osd max object name len = 460 2020-03-19T21:17:47.909 INFO:teuthology.orchestra.run.smithi170.stdout: debug client = 20 2020-03-19T21:17:47.909 INFO:teuthology.orchestra.run.smithi170.stdout: mon election default strategy = 3 2020-03-19T21:17:47.909 INFO:teuthology.orchestra.run.smithi170.stdout: fsid = 12d0a33b-6b40-4398-90b4-6b5df1da388e 2020-03-19T21:17:47.909 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:17:47.909 INFO:teuthology.orchestra.run.smithi170.stdout:[osd] 2020-03-19T21:17:47.909 INFO:teuthology.orchestra.run.smithi170.stdout: osd journal size = 100 2020-03-19T21:17:47.909 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:17:47.910 INFO:teuthology.orchestra.run.smithi170.stdout: osd scrub load threshold = 5.0 2020-03-19T21:17:47.910 INFO:teuthology.orchestra.run.smithi170.stdout: osd scrub max interval = 600 2020-03-19T21:17:47.910 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:17:47.910 INFO:teuthology.orchestra.run.smithi170.stdout: osd recover clone overlap = true 2020-03-19T21:17:47.910 INFO:teuthology.orchestra.run.smithi170.stdout: osd recovery max chunk = 1048576 2020-03-19T21:17:47.910 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:17:47.910 INFO:teuthology.orchestra.run.smithi170.stdout: osd debug shutdown = true 2020-03-19T21:17:47.911 INFO:teuthology.orchestra.run.smithi170.stdout: osd debug op order = true 2020-03-19T21:17:47.911 INFO:teuthology.orchestra.run.smithi170.stdout: osd debug verify stray on activate = true 2020-03-19T21:17:47.911 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:17:47.911 INFO:teuthology.orchestra.run.smithi170.stdout: osd open classes on start = true 2020-03-19T21:17:47.911 INFO:teuthology.orchestra.run.smithi170.stdout: osd debug pg log writeout = true 2020-03-19T21:17:47.911 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:17:47.911 INFO:teuthology.orchestra.run.smithi170.stdout: osd deep scrub update digest min age = 30 2020-03-19T21:17:47.912 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:17:47.912 INFO:teuthology.orchestra.run.smithi170.stdout: osd map max advance = 10 2020-03-19T21:17:47.912 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:17:47.912 INFO:teuthology.orchestra.run.smithi170.stdout: journal zero on create = true 2020-03-19T21:17:47.912 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:17:47.912 INFO:teuthology.orchestra.run.smithi170.stdout: filestore ondisk finisher threads = 3 2020-03-19T21:17:47.912 INFO:teuthology.orchestra.run.smithi170.stdout: filestore apply finisher threads = 3 2020-03-19T21:17:47.912 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:17:47.913 INFO:teuthology.orchestra.run.smithi170.stdout: bdev debug aio = true 2020-03-19T21:17:47.913 INFO:teuthology.orchestra.run.smithi170.stdout: osd debug misdirected ops = true 2020-03-19T21:17:47.913 INFO:teuthology.orchestra.run.smithi170.stdout: osd op queue cut off = debug_random 2020-03-19T21:17:47.913 INFO:teuthology.orchestra.run.smithi170.stdout: debug ms = 20 2020-03-19T21:17:47.913 INFO:teuthology.orchestra.run.smithi170.stdout: debug journal = 20 2020-03-19T21:17:47.913 INFO:teuthology.orchestra.run.smithi170.stdout: debug osd = 25 2020-03-19T21:17:47.913 INFO:teuthology.orchestra.run.smithi170.stdout: osd debug verify cached snaps = True 2020-03-19T21:17:47.914 INFO:teuthology.orchestra.run.smithi170.stdout: osd debug verify missing on start = True 2020-03-19T21:17:47.914 INFO:teuthology.orchestra.run.smithi170.stdout: osd op queue = debug_random 2020-03-19T21:17:47.914 INFO:teuthology.orchestra.run.smithi170.stdout: debug filestore = 20 2020-03-19T21:17:47.914 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:17:47.914 INFO:teuthology.orchestra.run.smithi170.stdout:[mgr] 2020-03-19T21:17:47.914 INFO:teuthology.orchestra.run.smithi170.stdout: debug ms = 1 2020-03-19T21:17:47.914 INFO:teuthology.orchestra.run.smithi170.stdout: debug mgr = 20 2020-03-19T21:17:47.915 INFO:teuthology.orchestra.run.smithi170.stdout: debug mon = 20 2020-03-19T21:17:47.915 INFO:teuthology.orchestra.run.smithi170.stdout: debug auth = 20 2020-03-19T21:17:47.915 INFO:teuthology.orchestra.run.smithi170.stdout: mon reweight min pgs per osd = 4 2020-03-19T21:17:47.915 INFO:teuthology.orchestra.run.smithi170.stdout: mon reweight min bytes per osd = 10 2020-03-19T21:17:47.915 INFO:teuthology.orchestra.run.smithi170.stdout: mgr/telemetry/nag = false 2020-03-19T21:17:47.915 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:17:47.915 INFO:teuthology.orchestra.run.smithi170.stdout:[mon] 2020-03-19T21:17:47.916 INFO:teuthology.orchestra.run.smithi170.stdout: debug ms = 1 2020-03-19T21:17:47.916 INFO:teuthology.orchestra.run.smithi170.stdout: debug mon = 20 2020-03-19T21:17:47.916 INFO:teuthology.orchestra.run.smithi170.stdout: debug paxos = 20 2020-03-19T21:17:47.916 INFO:teuthology.orchestra.run.smithi170.stdout: debug auth = 20 2020-03-19T21:17:47.916 INFO:teuthology.orchestra.run.smithi170.stdout: mon data avail warn = 5 2020-03-19T21:17:47.916 INFO:teuthology.orchestra.run.smithi170.stdout: mon mgr mkfs grace = 240 2020-03-19T21:17:47.916 INFO:teuthology.orchestra.run.smithi170.stdout: mon reweight min pgs per osd = 4 2020-03-19T21:17:47.917 INFO:teuthology.orchestra.run.smithi170.stdout: mon osd reporter subtree level = osd 2020-03-19T21:17:47.917 INFO:teuthology.orchestra.run.smithi170.stdout: mon osd prime pg temp = true 2020-03-19T21:17:47.917 INFO:teuthology.orchestra.run.smithi170.stdout: mon reweight min bytes per osd = 10 2020-03-19T21:17:47.917 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:17:47.917 INFO:teuthology.orchestra.run.smithi170.stdout:[client] 2020-03-19T21:17:47.917 INFO:teuthology.orchestra.run.smithi170.stdout: rgw cache enabled = true 2020-03-19T21:17:47.917 INFO:teuthology.orchestra.run.smithi170.stdout: rgw enable ops log = true 2020-03-19T21:17:47.918 INFO:teuthology.orchestra.run.smithi170.stdout: rgw enable usage log = true 2020-03-19T21:17:47.918 INFO:teuthology.orchestra.run.smithi170.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2020-03-19T21:17:47.918 INFO:teuthology.orchestra.run.smithi170.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2020-03-19T21:17:47.918 INFO:teuthology.orchestra.run.smithi170.stdout:[mon.a] 2020-03-19T21:17:47.918 INFO:teuthology.orchestra.run.smithi170.stdout:[mds.a] 2020-03-19T21:17:47.919 INFO:tasks.ceph:Creating admin key on mon.a... 2020-03-19T21:17:47.920 INFO:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2020-03-19T21:17:47.970 INFO:tasks.ceph:Copying monmap to all nodes... 2020-03-19T21:17:47.987 DEBUG:teuthology.orchestra.remote:smithi170:/etc/ceph/ceph.keyring is 206B 2020-03-19T21:17:48.086 DEBUG:teuthology.orchestra.remote:smithi170:/home/ubuntu/cephtest/ceph.monmap is 188B 2020-03-19T21:17:48.137 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi170.front.sepia.ceph.com 2020-03-19T21:17:48.137 INFO:teuthology.orchestra.run.smithi170:> sudo sh -c 'cat > /etc/ceph/ceph.keyring' && sudo chmod 0644 /etc/ceph/ceph.keyring 2020-03-19T21:17:48.196 INFO:teuthology.orchestra.run.smithi170:> cat > /home/ubuntu/cephtest/ceph.monmap 2020-03-19T21:17:48.281 INFO:tasks.ceph:Setting up mon nodes... 2020-03-19T21:17:48.281 INFO:tasks.ceph:Setting up mgr nodes... 2020-03-19T21:17:48.281 INFO:teuthology.orchestra.run.smithi170:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2020-03-19T21:17:48.374 INFO:teuthology.orchestra.run.smithi170.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2020-03-19T21:17:48.376 INFO:tasks.ceph:Setting up mds nodes... 2020-03-19T21:17:48.376 INFO:teuthology.orchestra.run.smithi170:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2020-03-19T21:17:48.429 INFO:teuthology.orchestra.run.smithi170.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2020-03-19T21:17:48.431 INFO:teuthology.orchestra.run.smithi170:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2020-03-19T21:17:48.458 INFO:tasks.ceph_client:Setting up client nodes... 2020-03-19T21:17:48.458 INFO:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2020-03-19T21:17:48.539 INFO:teuthology.orchestra.run.smithi170.stdout:creating /etc/ceph/ceph.client.0.keyring 2020-03-19T21:17:48.555 INFO:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2020-03-19T21:17:48.634 INFO:teuthology.orchestra.run.smithi170.stdout:creating /etc/ceph/ceph.client.1.keyring 2020-03-19T21:17:48.650 INFO:tasks.ceph:Running mkfs on osd nodes... 2020-03-19T21:17:48.650 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi170.front.sepia.ceph.com'): {'osd.1': '/dev/vg_nvme/lv_1', 'osd.0': '/dev/vg_nvme/lv_4', 'osd.2': '/dev/vg_nvme/lv_2'}} 2020-03-19T21:17:48.650 INFO:teuthology.orchestra.run.smithi170:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2020-03-19T21:17:48.717 INFO:tasks.ceph:{'osd.1': '/dev/vg_nvme/lv_1', 'osd.0': '/dev/vg_nvme/lv_4', 'osd.2': '/dev/vg_nvme/lv_2'} 2020-03-19T21:17:48.717 INFO:tasks.ceph:{} 2020-03-19T21:17:48.717 INFO:tasks.ceph:osd.0 2020-03-19T21:17:48.717 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi170.front.sepia.ceph.com 2020-03-19T21:17:48.718 INFO:teuthology.orchestra.run.smithi170:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2020-03-19T21:17:49.250 INFO:teuthology.orchestra.run.smithi170.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2020-03-19T21:17:49.251 INFO:teuthology.orchestra.run.smithi170.stdout: = sectsz=512 attr=2, projid32bit=1 2020-03-19T21:17:49.251 INFO:teuthology.orchestra.run.smithi170.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2020-03-19T21:17:49.251 INFO:teuthology.orchestra.run.smithi170.stdout: = reflink=1 2020-03-19T21:17:49.251 INFO:teuthology.orchestra.run.smithi170.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2020-03-19T21:17:49.251 INFO:teuthology.orchestra.run.smithi170.stdout: = sunit=0 swidth=0 blks 2020-03-19T21:17:49.252 INFO:teuthology.orchestra.run.smithi170.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2020-03-19T21:17:49.252 INFO:teuthology.orchestra.run.smithi170.stdout:log =internal log bsize=4096 blocks=11443, version=2 2020-03-19T21:17:49.252 INFO:teuthology.orchestra.run.smithi170.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2020-03-19T21:17:49.252 INFO:teuthology.orchestra.run.smithi170.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2020-03-19T21:17:49.253 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi170.front.sepia.ceph.com -o noatime 2020-03-19T21:17:49.253 INFO:teuthology.orchestra.run.smithi170:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-0 2020-03-19T21:17:49.349 INFO:teuthology.orchestra.run.smithi170:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2020-03-19T21:17:49.380 INFO:teuthology.orchestra.run.smithi170:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2020-03-19T21:17:49.447 INFO:tasks.ceph:{'osd.1': '/dev/vg_nvme/lv_1', 'osd.0': '/dev/vg_nvme/lv_4', 'osd.2': '/dev/vg_nvme/lv_2'} 2020-03-19T21:17:49.447 INFO:tasks.ceph:{} 2020-03-19T21:17:49.447 INFO:tasks.ceph:osd.1 2020-03-19T21:17:49.447 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi170.front.sepia.ceph.com 2020-03-19T21:17:49.447 INFO:teuthology.orchestra.run.smithi170:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2020-03-19T21:17:49.964 INFO:teuthology.orchestra.run.smithi170.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2020-03-19T21:17:49.964 INFO:teuthology.orchestra.run.smithi170.stdout: = sectsz=512 attr=2, projid32bit=1 2020-03-19T21:17:49.964 INFO:teuthology.orchestra.run.smithi170.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2020-03-19T21:17:49.965 INFO:teuthology.orchestra.run.smithi170.stdout: = reflink=1 2020-03-19T21:17:49.965 INFO:teuthology.orchestra.run.smithi170.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2020-03-19T21:17:49.965 INFO:teuthology.orchestra.run.smithi170.stdout: = sunit=0 swidth=0 blks 2020-03-19T21:17:49.965 INFO:teuthology.orchestra.run.smithi170.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2020-03-19T21:17:49.965 INFO:teuthology.orchestra.run.smithi170.stdout:log =internal log bsize=4096 blocks=11443, version=2 2020-03-19T21:17:49.965 INFO:teuthology.orchestra.run.smithi170.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2020-03-19T21:17:49.965 INFO:teuthology.orchestra.run.smithi170.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2020-03-19T21:17:49.966 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi170.front.sepia.ceph.com -o noatime 2020-03-19T21:17:49.966 INFO:teuthology.orchestra.run.smithi170:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-1 2020-03-19T21:17:50.016 INFO:teuthology.orchestra.run.smithi170:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2020-03-19T21:17:50.047 INFO:teuthology.orchestra.run.smithi170:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2020-03-19T21:17:50.114 INFO:tasks.ceph:{'osd.1': '/dev/vg_nvme/lv_1', 'osd.0': '/dev/vg_nvme/lv_4', 'osd.2': '/dev/vg_nvme/lv_2'} 2020-03-19T21:17:50.114 INFO:tasks.ceph:{} 2020-03-19T21:17:50.114 INFO:tasks.ceph:osd.2 2020-03-19T21:17:50.114 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi170.front.sepia.ceph.com 2020-03-19T21:17:50.114 INFO:teuthology.orchestra.run.smithi170:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2020-03-19T21:17:50.641 INFO:teuthology.orchestra.run.smithi170.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2020-03-19T21:17:50.642 INFO:teuthology.orchestra.run.smithi170.stdout: = sectsz=512 attr=2, projid32bit=1 2020-03-19T21:17:50.642 INFO:teuthology.orchestra.run.smithi170.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2020-03-19T21:17:50.642 INFO:teuthology.orchestra.run.smithi170.stdout: = reflink=1 2020-03-19T21:17:50.642 INFO:teuthology.orchestra.run.smithi170.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2020-03-19T21:17:50.642 INFO:teuthology.orchestra.run.smithi170.stdout: = sunit=0 swidth=0 blks 2020-03-19T21:17:50.642 INFO:teuthology.orchestra.run.smithi170.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2020-03-19T21:17:50.643 INFO:teuthology.orchestra.run.smithi170.stdout:log =internal log bsize=4096 blocks=11443, version=2 2020-03-19T21:17:50.643 INFO:teuthology.orchestra.run.smithi170.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2020-03-19T21:17:50.643 INFO:teuthology.orchestra.run.smithi170.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2020-03-19T21:17:50.644 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi170.front.sepia.ceph.com -o noatime 2020-03-19T21:17:50.644 INFO:teuthology.orchestra.run.smithi170:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-2 2020-03-19T21:17:50.722 INFO:teuthology.orchestra.run.smithi170:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2020-03-19T21:17:50.752 INFO:teuthology.orchestra.run.smithi170:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2020-03-19T21:17:50.834 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:50.833+0000 7f03b266eec0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2020-03-19T21:17:50.834 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:50.833+0000 7f03b266eec0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2020-03-19T21:17:50.835 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:50.833+0000 7f03b266eec0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2020-03-19T21:17:50.835 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:50.833+0000 7f03b266eec0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2020-03-19T21:17:50.835 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:50.833+0000 7f03b266eec0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2020-03-19T21:17:50.836 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:50.835+0000 7f03b266eec0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2020-03-19T21:17:52.653 INFO:teuthology.orchestra.run.smithi170:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2020-03-19T21:17:52.726 INFO:teuthology.orchestra.run.smithi170:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2020-03-19T21:17:52.814 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:52.813+0000 7f7869b8aec0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2020-03-19T21:17:52.814 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:52.813+0000 7f7869b8aec0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2020-03-19T21:17:52.814 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:52.813+0000 7f7869b8aec0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2020-03-19T21:17:52.815 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:52.813+0000 7f7869b8aec0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2020-03-19T21:17:52.815 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:52.813+0000 7f7869b8aec0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2020-03-19T21:17:52.815 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:52.813+0000 7f7869b8aec0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2020-03-19T21:17:54.631 INFO:teuthology.orchestra.run.smithi170:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2020-03-19T21:17:54.661 INFO:teuthology.orchestra.run.smithi170:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2020-03-19T21:17:54.746 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:54.744+0000 7f5ff4743ec0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2020-03-19T21:17:54.747 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:54.745+0000 7f5ff4743ec0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2020-03-19T21:17:54.747 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:54.745+0000 7f5ff4743ec0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2020-03-19T21:17:54.747 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:54.745+0000 7f5ff4743ec0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2020-03-19T21:17:54.747 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:54.745+0000 7f5ff4743ec0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2020-03-19T21:17:54.748 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:54.745+0000 7f5ff4743ec0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2020-03-19T21:17:56.563 INFO:teuthology.orchestra.run.smithi170:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2020-03-19T21:17:56.594 INFO:tasks.ceph:Reading keys from all nodes... 2020-03-19T21:17:56.595 INFO:teuthology.orchestra.run.smithi170:> mktemp 2020-03-19T21:17:56.652 INFO:teuthology.orchestra.run.smithi170.stdout:/tmp/tmp.lVLADSZdmK 2020-03-19T21:17:56.652 INFO:teuthology.orchestra.run.smithi170:> sudo cp /var/lib/ceph/mgr/ceph-x/keyring /tmp/tmp.lVLADSZdmK 2020-03-19T21:17:56.722 INFO:teuthology.orchestra.run.smithi170:> sudo chmod 0666 /tmp/tmp.lVLADSZdmK 2020-03-19T21:17:56.849 DEBUG:teuthology.orchestra.remote:smithi170:/tmp/tmp.lVLADSZdmK is 56B 2020-03-19T21:17:56.897 INFO:teuthology.orchestra.run.smithi170:> rm -fr /tmp/tmp.lVLADSZdmK 2020-03-19T21:17:56.911 INFO:teuthology.orchestra.run.smithi170:> mktemp 2020-03-19T21:17:56.965 INFO:teuthology.orchestra.run.smithi170.stdout:/tmp/tmp.pTzJnN5WTR 2020-03-19T21:17:56.965 INFO:teuthology.orchestra.run.smithi170:> sudo cp /var/lib/ceph/mds/ceph-a/keyring /tmp/tmp.pTzJnN5WTR 2020-03-19T21:17:57.032 INFO:teuthology.orchestra.run.smithi170:> sudo chmod 0666 /tmp/tmp.pTzJnN5WTR 2020-03-19T21:17:57.156 DEBUG:teuthology.orchestra.remote:smithi170:/tmp/tmp.pTzJnN5WTR is 56B 2020-03-19T21:17:57.205 INFO:teuthology.orchestra.run.smithi170:> rm -fr /tmp/tmp.pTzJnN5WTR 2020-03-19T21:17:57.219 INFO:teuthology.orchestra.run.smithi170:> mktemp 2020-03-19T21:17:57.271 INFO:teuthology.orchestra.run.smithi170.stdout:/tmp/tmp.egLA7L2KEz 2020-03-19T21:17:57.272 INFO:teuthology.orchestra.run.smithi170:> sudo cp /var/lib/ceph/osd/ceph-0/keyring /tmp/tmp.egLA7L2KEz 2020-03-19T21:17:57.338 INFO:teuthology.orchestra.run.smithi170:> sudo chmod 0666 /tmp/tmp.egLA7L2KEz 2020-03-19T21:17:57.462 DEBUG:teuthology.orchestra.remote:smithi170:/tmp/tmp.egLA7L2KEz is 56B 2020-03-19T21:17:57.510 INFO:teuthology.orchestra.run.smithi170:> rm -fr /tmp/tmp.egLA7L2KEz 2020-03-19T21:17:57.524 INFO:teuthology.orchestra.run.smithi170:> mktemp 2020-03-19T21:17:57.576 INFO:teuthology.orchestra.run.smithi170.stdout:/tmp/tmp.44XoWHkDBw 2020-03-19T21:17:57.577 INFO:teuthology.orchestra.run.smithi170:> sudo cp /var/lib/ceph/osd/ceph-1/keyring /tmp/tmp.44XoWHkDBw 2020-03-19T21:17:57.643 INFO:teuthology.orchestra.run.smithi170:> sudo chmod 0666 /tmp/tmp.44XoWHkDBw 2020-03-19T21:17:57.782 DEBUG:teuthology.orchestra.remote:smithi170:/tmp/tmp.44XoWHkDBw is 56B 2020-03-19T21:17:57.838 INFO:teuthology.orchestra.run.smithi170:> rm -fr /tmp/tmp.44XoWHkDBw 2020-03-19T21:17:57.858 INFO:teuthology.orchestra.run.smithi170:> mktemp 2020-03-19T21:17:57.904 INFO:teuthology.orchestra.run.smithi170.stdout:/tmp/tmp.Z4SYl5sbYL 2020-03-19T21:17:57.905 INFO:teuthology.orchestra.run.smithi170:> sudo cp /var/lib/ceph/osd/ceph-2/keyring /tmp/tmp.Z4SYl5sbYL 2020-03-19T21:17:57.972 INFO:teuthology.orchestra.run.smithi170:> sudo chmod 0666 /tmp/tmp.Z4SYl5sbYL 2020-03-19T21:17:58.095 DEBUG:teuthology.orchestra.remote:smithi170:/tmp/tmp.Z4SYl5sbYL is 56B 2020-03-19T21:17:58.144 INFO:teuthology.orchestra.run.smithi170:> rm -fr /tmp/tmp.Z4SYl5sbYL 2020-03-19T21:17:58.213 DEBUG:teuthology.orchestra.remote:smithi170:/etc/ceph/ceph.client.0.keyring is 59B 2020-03-19T21:17:58.306 DEBUG:teuthology.orchestra.remote:smithi170:/etc/ceph/ceph.client.1.keyring is 59B 2020-03-19T21:17:58.353 INFO:tasks.ceph:Adding keys to all mons... 2020-03-19T21:17:58.353 INFO:teuthology.orchestra.run.smithi170:> sudo tee -a /etc/ceph/ceph.keyring 2020-03-19T21:17:58.397 INFO:teuthology.orchestra.run.smithi170.stdout:[mgr.x] 2020-03-19T21:17:58.397 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQB84XNecLdFFhAAoUSv0pwv5v/fy1rWT0N4oA== 2020-03-19T21:17:58.397 INFO:teuthology.orchestra.run.smithi170.stdout:[mds.a] 2020-03-19T21:17:58.397 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQB84XNeZa6QGRAAsTxWXsTBgBPeW7mXEgfYkA== 2020-03-19T21:17:58.397 INFO:teuthology.orchestra.run.smithi170.stdout:[osd.0] 2020-03-19T21:17:58.398 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQB+4XNeqbSwMRAAtwoo94zPXvwZ3UHE6J5Ezw== 2020-03-19T21:17:58.398 INFO:teuthology.orchestra.run.smithi170.stdout:[osd.1] 2020-03-19T21:17:58.398 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQCA4XNeGBB+MBAA1xHIMw3hgh6XaViXX+EsKQ== 2020-03-19T21:17:58.398 INFO:teuthology.orchestra.run.smithi170.stdout:[osd.2] 2020-03-19T21:17:58.398 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQCC4XNeBG9sLBAAQrwEjhtizOIAmeCiLpbmwQ== 2020-03-19T21:17:58.398 INFO:teuthology.orchestra.run.smithi170.stdout:[client.0] 2020-03-19T21:17:58.398 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQB84XNezQUbIBAA74W6M4XM64oDyaHVaaiKHA== 2020-03-19T21:17:58.399 INFO:teuthology.orchestra.run.smithi170.stdout:[client.1] 2020-03-19T21:17:58.399 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQB84XNe8jrPJRAAW8sWRQVYMjFejOKV0guGOw== 2020-03-19T21:17:58.400 INFO:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mds 'allow *' --cap osd 'allow *' --cap mon 'allow profile mgr' 2020-03-19T21:17:58.487 INFO:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mds allow --cap osd 'allow *' --cap mgr 'allow *' --cap mon 'allow *' 2020-03-19T21:17:58.538 INFO:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mgr 'allow *' --cap osd 'allow *' --cap mon 'allow *' 2020-03-19T21:17:58.586 INFO:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mgr 'allow *' --cap osd 'allow *' --cap mon 'allow *' 2020-03-19T21:17:58.633 INFO:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mgr 'allow *' --cap osd 'allow *' --cap mon 'allow *' 2020-03-19T21:17:58.680 INFO:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mds allow --cap osd 'allow rwx' --cap mgr 'allow r' --cap mon 'allow rw' 2020-03-19T21:17:58.729 INFO:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mds allow --cap osd 'allow rwx' --cap mgr 'allow r' --cap mon 'allow rw' 2020-03-19T21:17:58.779 INFO:tasks.ceph:Running mkfs on mon nodes... 2020-03-19T21:17:58.779 INFO:teuthology.orchestra.run.smithi170:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2020-03-19T21:17:58.806 INFO:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2020-03-19T21:17:58.898 INFO:teuthology.orchestra.run.smithi170:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2020-03-19T21:17:58.925 INFO:teuthology.orchestra.run.smithi170:> rm -- /home/ubuntu/cephtest/ceph.monmap 2020-03-19T21:17:58.979 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2020-03-19T21:17:58.979 INFO:tasks.ceph.mon.a:Restarting daemon 2020-03-19T21:17:58.979 INFO:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2020-03-19T21:17:59.021 INFO:tasks.ceph.mon.a:Started 2020-03-19T21:17:59.021 INFO:teuthology.orchestra.run.smithi170:> sudo ceph config set mgr mgr/devicehealth/enable_monitoring false --force 2020-03-19T21:17:59.155 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.128+0000 7fc1169f6700 1 Processor -- start 2020-03-19T21:17:59.155 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.128+0000 7fc1169f6700 1 -- start start 2020-03-19T21:17:59.155 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.129+0000 7fc1169f6700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fc11007e780 0x7fc11008f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T21:17:59.155 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.129+0000 7fc1169f6700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc11005c700 con 0x7fc11008f680 2020-03-19T21:17:59.155 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.129+0000 7fc1169f6700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc110090520 con 0x7fc11007e780 2020-03-19T21:17:59.156 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.129+0000 7fc114f93700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fc11007e780 0x7fc11008f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:17:59.156 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.129+0000 7fc10ffff700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fc11008f680 0x7fc11008fb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:53356/0 (socket says 172.21.15.170:53356) 2020-03-19T21:17:59.156 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.129+0000 7fc10ffff700 1 -- 172.21.15.170:0/406642666 learned_addr learned my addr 172.21.15.170:0/406642666 (peer_addr_for_me v1:172.21.15.170:0/0) 2020-03-19T21:17:59.156 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.129+0000 7fc114f93700 1 -- 172.21.15.170:0/406642666 >> v2:172.21.15.170:3300/0 conn(0x7fc11007e780 msgr2=0x7fc11008f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 12 2020-03-19T21:17:59.156 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.129+0000 7fc114f93700 1 -- 172.21.15.170:0/406642666 >> v2:172.21.15.170:3300/0 conn(0x7fc11007e780 msgr2=0x7fc11008f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-03-19T21:17:59.156 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.129+0000 7fc114f93700 1 --2- 172.21.15.170:0/406642666 >> v2:172.21.15.170:3300/0 conn(0x7fc11007e780 0x7fc11008f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).handle_read_frame_preamble_main read frame length and tag failed r=-1 ((1) Operation not permitted) 2020-03-19T21:17:59.156 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.129+0000 7fc114f93700 1 --2- 172.21.15.170:0/406642666 >> v2:172.21.15.170:3300/0 conn(0x7fc11007e780 0x7fc11008f220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rx=0 tx=0)._fault waiting 0.200000 2020-03-19T21:17:59.157 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.149+0000 7fc10f7fe700 1 -- 172.21.15.170:0/406642666 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1040778139 0 0) 0x7fc11005c700 con 0x7fc11008f680 2020-03-19T21:17:59.157 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.149+0000 7fc10f7fe700 1 -- 172.21.15.170:0/406642666 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc0fc003230 con 0x7fc11008f680 2020-03-19T21:17:59.157 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.149+0000 7fc10f7fe700 1 -- 172.21.15.170:0/406642666 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2088596014 0 0) 0x7fc0fc003230 con 0x7fc11008f680 2020-03-19T21:17:59.157 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.149+0000 7fc10f7fe700 1 -- 172.21.15.170:0/406642666 >> v2:172.21.15.170:3300/0 conn(0x7fc11007e780 msgr2=0x7fc11008f220 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-03-19T21:17:59.157 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.149+0000 7fc10f7fe700 1 --2- 172.21.15.170:0/406642666 >> v2:172.21.15.170:3300/0 conn(0x7fc11007e780 0x7fc11008f220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-03-19T21:17:59.157 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.149+0000 7fc10f7fe700 1 -- 172.21.15.170:0/406642666 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc11007eb80 con 0x7fc11008f680 2020-03-19T21:17:59.157 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.149+0000 7fc10f7fe700 1 -- 172.21.15.170:0/406642666 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (unknown 2983176748 0 0) 0x7fc0f8001610 con 0x7fc11008f680 2020-03-19T21:17:59.158 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.149+0000 7fc10f7fe700 1 -- 172.21.15.170:0/406642666 >> v1:172.21.15.170:6789/0 conn(0x7fc11008f680 legacy=0x7fc11008fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:17:59.158 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.149+0000 7fc10f7fe700 1 --2- 172.21.15.170:0/406642666 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc0fc003720 0x7fc0fc003b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T21:17:59.158 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.149+0000 7fc10f7fe700 1 -- 172.21.15.170:0/406642666 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0fc004040 con 0x7fc0fc003720 2020-03-19T21:17:59.158 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.149+0000 7fc114f93700 1 --2- 172.21.15.170:0/406642666 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc0fc003720 0x7fc0fc003b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:17:59.158 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.150+0000 7fc114f93700 1 -- 172.21.15.170:0/406642666 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0fc003230 con 0x7fc0fc003720 2020-03-19T21:17:59.158 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.150+0000 7fc114f93700 1 --2- 172.21.15.170:0/406642666 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc0fc003720 0x7fc0fc003b20 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7fc108002f30 tx=0x7fc108053f90).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:17:59.158 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.150+0000 7fc114f93700 1 --2- 172.21.15.170:0/406642666 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc0fc003720 0x7fc0fc003b20 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7fc108002f30 tx=0x7fc108053f90).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:17:59.159 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.150+0000 7fc114f93700 1 --2- 172.21.15.170:0/406642666 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc0fc003720 0x7fc0fc003b20 secure :-1 s=READY pgs=2 cs=0 l=1 rx=0x7fc108002f30 tx=0x7fc108053f90).ready entity=mon.0 client_cookie=bef61b2d230ef4b9 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:17:59.159 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.150+0000 7fc114f93700 1 --2- 172.21.15.170:0/406642666 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc0fc003720 0x7fc0fc003b20 secure :-1 s=THROTTLE_DONE pgs=2 cs=0 l=1 rx=0x7fc108002f30 tx=0x7fc108053f90).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:17:59.159 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.150+0000 7fc10f7fe700 1 -- 172.21.15.170:0/406642666 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fc108055630 con 0x7fc0fc003720 2020-03-19T21:17:59.160 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.150+0000 7fc114f93700 1 --2- 172.21.15.170:0/406642666 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc0fc003720 0x7fc0fc003b20 secure :-1 s=THROTTLE_DONE pgs=2 cs=0 l=1 rx=0x7fc108002f30 tx=0x7fc108053f90).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:17:59.160 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.150+0000 7fc114f93700 1 --2- 172.21.15.170:0/406642666 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc0fc003720 0x7fc0fc003b20 secure :-1 s=THROTTLE_DONE pgs=2 cs=0 l=1 rx=0x7fc108002f30 tx=0x7fc108053f90).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:17:59.160 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.150+0000 7fc10f7fe700 1 -- 172.21.15.170:0/406642666 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc108055790 con 0x7fc0fc003720 2020-03-19T21:17:59.160 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.150+0000 7fc10f7fe700 1 -- 172.21.15.170:0/406642666 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fc108055ab0 con 0x7fc0fc003720 2020-03-19T21:17:59.161 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.150+0000 7fc1169f6700 1 -- 172.21.15.170:0/406642666 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc0fc003720 msgr2=0x7fc0fc003b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:17:59.161 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.150+0000 7fc1169f6700 1 --2- 172.21.15.170:0/406642666 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc0fc003720 0x7fc0fc003b20 secure :-1 s=READY pgs=2 cs=0 l=1 rx=0x7fc108002f30 tx=0x7fc108053f90).stop 2020-03-19T21:17:59.161 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.150+0000 7fc1169f6700 1 -- 172.21.15.170:0/406642666 shutdown_connections 2020-03-19T21:17:59.162 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.150+0000 7fc1169f6700 1 --2- 172.21.15.170:0/406642666 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc0fc003720 0x7fc0fc003b20 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:17:59.162 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.150+0000 7fc1169f6700 1 --2- 172.21.15.170:0/406642666 >> v2:172.21.15.170:3300/0 conn(0x7fc11007e780 0x7fc11008f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-03-19T21:17:59.162 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.150+0000 7fc1169f6700 1 -- 172.21.15.170:0/406642666 >> 172.21.15.170:0/406642666 conn(0x7fc110095460 msgr2=0x7fc110091320 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:17:59.162 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.151+0000 7fc1169f6700 1 -- 172.21.15.170:0/406642666 shutdown_connections 2020-03-19T21:17:59.162 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.151+0000 7fc1169f6700 1 -- 172.21.15.170:0/406642666 wait complete. 2020-03-19T21:17:59.162 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.151+0000 7fc1169f6700 1 Processor -- start 2020-03-19T21:17:59.163 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.151+0000 7fc1169f6700 1 -- start start 2020-03-19T21:17:59.163 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.152+0000 7fc1169f6700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fc11007e780 0x7fc110110140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:17:59.163 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.152+0000 7fc1169f6700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc110111220 con 0x7fc0fc003720 2020-03-19T21:17:59.163 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.152+0000 7fc1169f6700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc110112500 con 0x7fc11007e780 2020-03-19T21:17:59.163 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.152+0000 7fc114f93700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fc0fc003720 0x7fc11010da30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:53360/0 (socket says 172.21.15.170:53360) 2020-03-19T21:17:59.163 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.152+0000 7fc114f93700 1 -- 172.21.15.170:0/4044035290 learned_addr learned my addr 172.21.15.170:0/4044035290 (peer_addr_for_me v1:172.21.15.170:0/0) 2020-03-19T21:17:59.163 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.152+0000 7fc10ffff700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fc11007e780 0x7fc110110140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:17:59.164 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.152+0000 7fc10d7fa700 1 -- 172.21.15.170:0/4044035290 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1016301882 0 0) 0x7fc110111220 con 0x7fc0fc003720 2020-03-19T21:17:59.164 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.152+0000 7fc10d7fa700 1 -- 172.21.15.170:0/4044035290 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc0f4003230 con 0x7fc0fc003720 2020-03-19T21:17:59.164 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.152+0000 7fc10ffff700 1 -- 172.21.15.170:0/4044035290 >> v1:172.21.15.170:6789/0 conn(0x7fc0fc003720 legacy=0x7fc11010da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:17:59.164 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.152+0000 7fc10ffff700 1 -- 172.21.15.170:0/4044035290 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc11010f940 con 0x7fc11007e780 2020-03-19T21:17:59.164 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.152+0000 7fc10ffff700 1 --2- 172.21.15.170:0/4044035290 >> v2:172.21.15.170:3300/0 conn(0x7fc11007e780 0x7fc110110140 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7fc0f80013e0 tx=0x7fc11005ee50).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:17:59.164 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.152+0000 7fc10ffff700 1 --2- 172.21.15.170:0/4044035290 >> v2:172.21.15.170:3300/0 conn(0x7fc11007e780 0x7fc110110140 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7fc0f80013e0 tx=0x7fc11005ee50).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:17:59.164 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.152+0000 7fc10ffff700 1 --2- 172.21.15.170:0/4044035290 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc11007e780 0x7fc110110140 secure :-1 s=READY pgs=4 cs=0 l=1 rx=0x7fc0f80013e0 tx=0x7fc11005ee50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:17:59.164 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.153+0000 7fc10ffff700 1 --2- 172.21.15.170:0/4044035290 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc11007e780 0x7fc110110140 secure :-1 s=THROTTLE_DONE pgs=4 cs=0 l=1 rx=0x7fc0f80013e0 tx=0x7fc11005ee50).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:17:59.165 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.153+0000 7fc10d7fa700 1 -- 172.21.15.170:0/4044035290 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fc0f800c870 con 0x7fc11007e780 2020-03-19T21:17:59.165 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.153+0000 7fc10ffff700 1 --2- 172.21.15.170:0/4044035290 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc11007e780 0x7fc110110140 secure :-1 s=THROTTLE_DONE pgs=4 cs=0 l=1 rx=0x7fc0f80013e0 tx=0x7fc11005ee50).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:17:59.165 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.153+0000 7fc10ffff700 1 --2- 172.21.15.170:0/4044035290 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc11007e780 0x7fc110110140 secure :-1 s=THROTTLE_DONE pgs=4 cs=0 l=1 rx=0x7fc0f80013e0 tx=0x7fc11005ee50).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:17:59.165 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.153+0000 7fc10d7fa700 1 -- 172.21.15.170:0/4044035290 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0f800c9d0 con 0x7fc11007e780 2020-03-19T21:17:59.165 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.153+0000 7fc1169f6700 1 -- 172.21.15.170:0/4044035290 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc11010f410 con 0x7fc11007e780 2020-03-19T21:17:59.165 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.153+0000 7fc10d7fa700 1 -- 172.21.15.170:0/4044035290 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fc0f800ccf0 con 0x7fc11007e780 2020-03-19T21:17:59.166 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.153+0000 7fc1169f6700 1 -- 172.21.15.170:0/4044035290 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc11010e140 con 0x7fc11007e780 2020-03-19T21:17:59.166 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.153+0000 7fc10ffff700 1 --2- 172.21.15.170:0/4044035290 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc11007e780 0x7fc110110140 secure :-1 s=THROTTLE_DONE pgs=4 cs=0 l=1 rx=0x7fc0f80013e0 tx=0x7fc11005ee50).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:17:59.166 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.153+0000 7fc10d7fa700 1 -- 172.21.15.170:0/4044035290 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 1) v1 ==== 360+0+0 (secure 0 0 0) 0x7fc0f80048b0 con 0x7fc11007e780 2020-03-19T21:17:59.166 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.153+0000 7fc10ffff700 1 --2- 172.21.15.170:0/4044035290 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc11007e780 0x7fc110110140 secure :-1 s=THROTTLE_DONE pgs=4 cs=0 l=1 rx=0x7fc0f80013e0 tx=0x7fc11005ee50).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:17:59.167 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.153+0000 7fc10d7fa700 1 -- 172.21.15.170:0/4044035290 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 754+0+0 (secure 0 0 0) 0x7fc0f80011f0 con 0x7fc11007e780 2020-03-19T21:17:59.167 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.154+0000 7fc1169f6700 1 -- 172.21.15.170:0/4044035290 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc11010eb70 con 0x7fc11007e780 2020-03-19T21:17:59.167 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.156+0000 7fc10ffff700 1 --2- 172.21.15.170:0/4044035290 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc11007e780 0x7fc110110140 secure :-1 s=THROTTLE_DONE pgs=4 cs=0 l=1 rx=0x7fc0f80013e0 tx=0x7fc11005ee50).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:17:59.167 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.156+0000 7fc10d7fa700 1 -- 172.21.15.170:0/4044035290 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+68559 (secure 0 0 0) 0x7fc0f80101c0 con 0x7fc11007e780 2020-03-19T21:17:59.227 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.225+0000 7fc1169f6700 1 -- 172.21.15.170:0/4044035290 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "config set", "who": "mgr", "name": "mgr/devicehealth/enable_monitoring", "value": "false", "force": true} v 0) v1 -- 0x7fc110056db0 con 0x7fc11007e780 2020-03-19T21:17:59.228 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.227+0000 7fc10ffff700 1 --2- 172.21.15.170:0/4044035290 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc11007e780 0x7fc110110140 secure :-1 s=THROTTLE_DONE pgs=4 cs=0 l=1 rx=0x7fc0f80013e0 tx=0x7fc11005ee50).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:17:59.229 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.227+0000 7fc10d7fa700 1 -- 172.21.15.170:0/4044035290 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "config set", "who": "mgr", "name": "mgr/devicehealth/enable_monitoring", "value": "false", "force": true}]=0 v2) v1 ==== 151+0+0 (secure 0 0 0) 0x7fc0f8010360 con 0x7fc11007e780 2020-03-19T21:17:59.230 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.228+0000 7fc1169f6700 1 -- 172.21.15.170:0/4044035290 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc11007e780 msgr2=0x7fc110110140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:17:59.230 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.228+0000 7fc1169f6700 1 --2- 172.21.15.170:0/4044035290 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc11007e780 0x7fc110110140 secure :-1 s=READY pgs=4 cs=0 l=1 rx=0x7fc0f80013e0 tx=0x7fc11005ee50).stop 2020-03-19T21:17:59.230 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.228+0000 7fc1169f6700 1 -- 172.21.15.170:0/4044035290 shutdown_connections 2020-03-19T21:17:59.230 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.228+0000 7fc1169f6700 1 --2- 172.21.15.170:0/4044035290 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc11007e780 0x7fc110110140 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:17:59.230 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.228+0000 7fc1169f6700 1 -- 172.21.15.170:0/4044035290 >> 172.21.15.170:0/4044035290 conn(0x7fc110095460 msgr2=0x7fc110062dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:17:59.230 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.228+0000 7fc1169f6700 1 -- 172.21.15.170:0/4044035290 shutdown_connections 2020-03-19T21:17:59.231 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.228+0000 7fc1169f6700 1 -- 172.21.15.170:0/4044035290 wait complete. 2020-03-19T21:17:59.239 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2020-03-19T21:17:59.239 INFO:tasks.ceph.mgr.x:Restarting daemon 2020-03-19T21:17:59.240 INFO:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2020-03-19T21:17:59.243 INFO:tasks.ceph.mgr.x:Started 2020-03-19T21:17:59.243 INFO:tasks.ceph:Setting crush tunables to default 2020-03-19T21:17:59.244 INFO:teuthology.orchestra.run.smithi170:> sudo ceph --cluster ceph osd crush tunables default 2020-03-19T21:17:59.356 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.348+0000 7f5767686700 1 Processor -- start 2020-03-19T21:17:59.356 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.348+0000 7f5767686700 1 -- start start 2020-03-19T21:17:59.357 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.349+0000 7f5767686700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f576008c7d0 0x7f5760087cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T21:17:59.357 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.349+0000 7f5767686700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5760090f70 con 0x7f576008bce0 2020-03-19T21:17:59.357 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.349+0000 7f5767686700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57600904e0 con 0x7f576008c7d0 2020-03-19T21:17:59.357 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.349+0000 7f5765422700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f576008c7d0 0x7f5760087cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:17:59.357 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.349+0000 7f5765c23700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f576008bce0 0x7f576008c0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:53372/0 (socket says 172.21.15.170:53372) 2020-03-19T21:17:59.357 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.349+0000 7f5765c23700 1 -- 172.21.15.170:0/158913778 learned_addr learned my addr 172.21.15.170:0/158913778 (peer_addr_for_me v1:172.21.15.170:0/0) 2020-03-19T21:17:59.358 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.349+0000 7f5765422700 1 -- 172.21.15.170:0/158913778 >> v1:172.21.15.170:6789/0 conn(0x7f576008bce0 legacy=0x7f576008c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:17:59.358 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.349+0000 7f5764c21700 1 -- 172.21.15.170:0/158913778 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1537615857 0 0) 0x7f5760090f70 con 0x7f576008bce0 2020-03-19T21:17:59.358 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.349+0000 7f5765422700 1 -- 172.21.15.170:0/158913778 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57600908e0 con 0x7f576008c7d0 2020-03-19T21:17:59.358 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.349+0000 7f5765422700 1 --2- 172.21.15.170:0/158913778 >> v2:172.21.15.170:3300/0 conn(0x7f576008c7d0 0x7f5760087cb0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f575c002f30 tx=0x7f575c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:17:59.358 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.349+0000 7f5765422700 1 --2- 172.21.15.170:0/158913778 >> v2:172.21.15.170:3300/0 conn(0x7f576008c7d0 0x7f5760087cb0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f575c002f30 tx=0x7f575c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:17:59.358 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.349+0000 7f5765422700 1 --2- 172.21.15.170:0/158913778 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f576008c7d0 0x7f5760087cb0 secure :-1 s=READY pgs=10 cs=0 l=1 rx=0x7f575c002f30 tx=0x7f575c007a40).ready entity=mon.0 client_cookie=1e5669087cba16ba server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:17:59.359 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.349+0000 7f5765422700 1 --2- 172.21.15.170:0/158913778 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f576008c7d0 0x7f5760087cb0 secure :-1 s=THROTTLE_DONE pgs=10 cs=0 l=1 rx=0x7f575c002f30 tx=0x7f575c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:17:59.359 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.349+0000 7f5764c21700 1 -- 172.21.15.170:0/158913778 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f575c008e80 con 0x7f576008c7d0 2020-03-19T21:17:59.359 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.350+0000 7f5765422700 1 --2- 172.21.15.170:0/158913778 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f576008c7d0 0x7f5760087cb0 secure :-1 s=THROTTLE_DONE pgs=10 cs=0 l=1 rx=0x7f575c002f30 tx=0x7f575c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:17:59.359 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.350+0000 7f5764c21700 1 -- 172.21.15.170:0/158913778 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f575c0113f0 con 0x7f576008c7d0 2020-03-19T21:17:59.359 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.350+0000 7f5765422700 1 --2- 172.21.15.170:0/158913778 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f576008c7d0 0x7f5760087cb0 secure :-1 s=THROTTLE_DONE pgs=10 cs=0 l=1 rx=0x7f575c002f30 tx=0x7f575c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:17:59.360 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.350+0000 7f5767686700 1 -- 172.21.15.170:0/158913778 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f576008c7d0 msgr2=0x7f5760087cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:17:59.360 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.350+0000 7f5764c21700 1 -- 172.21.15.170:0/158913778 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f575c0116d0 con 0x7f576008c7d0 2020-03-19T21:17:59.360 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.350+0000 7f5767686700 1 --2- 172.21.15.170:0/158913778 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f576008c7d0 0x7f5760087cb0 secure :-1 s=READY pgs=10 cs=0 l=1 rx=0x7f575c002f30 tx=0x7f575c007a40).stop 2020-03-19T21:17:59.360 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.350+0000 7f5767686700 1 -- 172.21.15.170:0/158913778 shutdown_connections 2020-03-19T21:17:59.360 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.350+0000 7f5767686700 1 --2- 172.21.15.170:0/158913778 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f576008c7d0 0x7f5760087cb0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:17:59.361 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.350+0000 7f5767686700 1 -- 172.21.15.170:0/158913778 >> 172.21.15.170:0/158913778 conn(0x7f57600951d0 msgr2=0x7f5760091250 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:17:59.362 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.350+0000 7f5767686700 1 -- 172.21.15.170:0/158913778 shutdown_connections 2020-03-19T21:17:59.362 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.350+0000 7f5767686700 1 -- 172.21.15.170:0/158913778 wait complete. 2020-03-19T21:17:59.362 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.351+0000 7f5767686700 1 Processor -- start 2020-03-19T21:17:59.362 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.351+0000 7f5767686700 1 -- start start 2020-03-19T21:17:59.362 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.351+0000 7f5767686700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f576008c7d0 0x7f576008b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:17:59.362 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.351+0000 7f5767686700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5760093870 con 0x7f576008bce0 2020-03-19T21:17:59.362 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.351+0000 7f5767686700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f576008b560 con 0x7f576008c7d0 2020-03-19T21:17:59.363 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.352+0000 7f5765c23700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f576008bce0 0x7f576008a9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:53376/0 (socket says 172.21.15.170:53376) 2020-03-19T21:17:59.363 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.352+0000 7f5765c23700 1 -- 172.21.15.170:0/2218456772 learned_addr learned my addr 172.21.15.170:0/2218456772 (peer_addr_for_me v1:172.21.15.170:0/0) 2020-03-19T21:17:59.363 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.352+0000 7f5765422700 1 --2- 172.21.15.170:0/2218456772 >> v2:172.21.15.170:3300/0 conn(0x7f576008c7d0 0x7f576008b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:17:59.363 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.352+0000 7f57567fc700 1 -- 172.21.15.170:0/2218456772 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2271108693 0 0) 0x7f5760093870 con 0x7f576008bce0 2020-03-19T21:17:59.364 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.352+0000 7f57567fc700 1 -- 172.21.15.170:0/2218456772 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5748003230 con 0x7f576008bce0 2020-03-19T21:17:59.364 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.352+0000 7f5765422700 1 -- 172.21.15.170:0/2218456772 >> v1:172.21.15.170:6789/0 conn(0x7f576008bce0 legacy=0x7f576008a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:17:59.364 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.352+0000 7f5765422700 1 -- 172.21.15.170:0/2218456772 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5760071450 con 0x7f576008c7d0 2020-03-19T21:17:59.364 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.352+0000 7f5765422700 1 --2- 172.21.15.170:0/2218456772 >> v2:172.21.15.170:3300/0 conn(0x7f576008c7d0 0x7f576008b100 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f575c002f30 tx=0x7f575c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:17:59.364 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.352+0000 7f5765422700 1 --2- 172.21.15.170:0/2218456772 >> v2:172.21.15.170:3300/0 conn(0x7f576008c7d0 0x7f576008b100 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f575c002f30 tx=0x7f575c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:17:59.364 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.352+0000 7f5765422700 1 --2- 172.21.15.170:0/2218456772 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f576008c7d0 0x7f576008b100 secure :-1 s=READY pgs=12 cs=0 l=1 rx=0x7f575c002f30 tx=0x7f575c007a40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:17:59.364 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.352+0000 7f5765422700 1 --2- 172.21.15.170:0/2218456772 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f576008c7d0 0x7f576008b100 secure :-1 s=THROTTLE_DONE pgs=12 cs=0 l=1 rx=0x7f575c002f30 tx=0x7f575c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:17:59.365 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.352+0000 7f57567fc700 1 -- 172.21.15.170:0/2218456772 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f575c00f830 con 0x7f576008c7d0 2020-03-19T21:17:59.365 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.352+0000 7f5765422700 1 --2- 172.21.15.170:0/2218456772 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f576008c7d0 0x7f576008b100 secure :-1 s=THROTTLE_DONE pgs=12 cs=0 l=1 rx=0x7f575c002f30 tx=0x7f575c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:17:59.365 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.352+0000 7f5765422700 1 --2- 172.21.15.170:0/2218456772 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f576008c7d0 0x7f576008b100 secure :-1 s=THROTTLE_DONE pgs=12 cs=0 l=1 rx=0x7f575c002f30 tx=0x7f575c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:17:59.365 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.352+0000 7f57567fc700 1 -- 172.21.15.170:0/2218456772 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f575c011e20 con 0x7f576008c7d0 2020-03-19T21:17:59.365 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.352+0000 7f5767686700 1 -- 172.21.15.170:0/2218456772 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5760077490 con 0x7f576008c7d0 2020-03-19T21:17:59.365 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.352+0000 7f57567fc700 1 -- 172.21.15.170:0/2218456772 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f575c007280 con 0x7f576008c7d0 2020-03-19T21:17:59.366 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.352+0000 7f5765422700 1 --2- 172.21.15.170:0/2218456772 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f576008c7d0 0x7f576008b100 secure :-1 s=THROTTLE_DONE pgs=12 cs=0 l=1 rx=0x7f575c002f30 tx=0x7f575c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:17:59.366 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.352+0000 7f5767686700 1 -- 172.21.15.170:0/2218456772 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5760071210 con 0x7f576008c7d0 2020-03-19T21:17:59.366 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.352+0000 7f57567fc700 1 -- 172.21.15.170:0/2218456772 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 1) v1 ==== 360+0+0 (secure 0 0 0) 0x7f575c0058a0 con 0x7f576008c7d0 2020-03-19T21:17:59.366 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.352+0000 7f5765422700 1 --2- 172.21.15.170:0/2218456772 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f576008c7d0 0x7f576008b100 secure :-1 s=THROTTLE_DONE pgs=12 cs=0 l=1 rx=0x7f575c002f30 tx=0x7f575c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:17:59.366 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.352+0000 7f57567fc700 1 -- 172.21.15.170:0/2218456772 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 754+0+0 (secure 0 0 0) 0x7f575c005b30 con 0x7f576008c7d0 2020-03-19T21:17:59.366 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.353+0000 7f5767686700 1 -- 172.21.15.170:0/2218456772 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5760076380 con 0x7f576008c7d0 2020-03-19T21:17:59.367 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.355+0000 7f5765422700 1 --2- 172.21.15.170:0/2218456772 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f576008c7d0 0x7f576008b100 secure :-1 s=THROTTLE_DONE pgs=12 cs=0 l=1 rx=0x7f575c002f30 tx=0x7f575c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:17:59.367 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.355+0000 7f57567fc700 1 -- 172.21.15.170:0/2218456772 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+68559 (secure 0 0 0) 0x7f5760071210 con 0x7f576008c7d0 2020-03-19T21:17:59.435 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.433+0000 7f5767686700 1 -- 172.21.15.170:0/2218456772 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f5760056db0 con 0x7f576008c7d0 2020-03-19T21:17:59.438 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.435+0000 7f5765422700 1 --2- 172.21.15.170:0/2218456772 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f576008c7d0 0x7f576008b100 secure :-1 s=THROTTLE_DONE pgs=12 cs=0 l=1 rx=0x7f575c002f30 tx=0x7f575c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:17:59.438 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.435+0000 7f57567fc700 1 -- 172.21.15.170:0/2218456772 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f575c00bb30 con 0x7f576008c7d0 2020-03-19T21:17:59.439 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.436+0000 7f5767686700 1 -- 172.21.15.170:0/2218456772 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f576008c7d0 msgr2=0x7f576008b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:17:59.439 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.436+0000 7f5767686700 1 --2- 172.21.15.170:0/2218456772 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f576008c7d0 0x7f576008b100 secure :-1 s=READY pgs=12 cs=0 l=1 rx=0x7f575c002f30 tx=0x7f575c007a40).stop 2020-03-19T21:17:59.439 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.436+0000 7f5767686700 1 -- 172.21.15.170:0/2218456772 shutdown_connections 2020-03-19T21:17:59.439 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.436+0000 7f5767686700 1 --2- 172.21.15.170:0/2218456772 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f576008c7d0 0x7f576008b100 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:17:59.440 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.437+0000 7f5767686700 1 -- 172.21.15.170:0/2218456772 >> 172.21.15.170:0/2218456772 conn(0x7f57600951d0 msgr2=0x7f5760062870 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:17:59.440 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.437+0000 7f5767686700 1 -- 172.21.15.170:0/2218456772 shutdown_connections 2020-03-19T21:17:59.440 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:17:59.437+0000 7f5767686700 1 -- 172.21.15.170:0/2218456772 wait complete. 2020-03-19T21:17:59.440 INFO:teuthology.orchestra.run.smithi170.stderr:adjusted tunables profile to default 2020-03-19T21:17:59.452 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2020-03-19T21:17:59.453 INFO:teuthology.orchestra.run.smithi170:> mktemp 2020-03-19T21:17:59.467 INFO:teuthology.orchestra.run.smithi170.stdout:/tmp/tmp.1VKbpQreOA 2020-03-19T21:17:59.467 INFO:teuthology.orchestra.run.smithi170:> sudo cp /var/lib/ceph/osd/ceph-0/fsid /tmp/tmp.1VKbpQreOA 2020-03-19T21:17:59.535 INFO:teuthology.orchestra.run.smithi170:> sudo chmod 0666 /tmp/tmp.1VKbpQreOA 2020-03-19T21:17:59.659 DEBUG:teuthology.orchestra.remote:smithi170:/tmp/tmp.1VKbpQreOA is 37B 2020-03-19T21:17:59.707 INFO:teuthology.orchestra.run.smithi170:> rm -fr /tmp/tmp.1VKbpQreOA 2020-03-19T21:17:59.721 INFO:teuthology.orchestra.run.smithi170:> mktemp 2020-03-19T21:17:59.775 INFO:teuthology.orchestra.run.smithi170.stdout:/tmp/tmp.alx1SYOXA1 2020-03-19T21:17:59.775 INFO:teuthology.orchestra.run.smithi170:> sudo cp /var/lib/ceph/osd/ceph-1/fsid /tmp/tmp.alx1SYOXA1 2020-03-19T21:17:59.842 INFO:teuthology.orchestra.run.smithi170:> sudo chmod 0666 /tmp/tmp.alx1SYOXA1 2020-03-19T21:17:59.966 DEBUG:teuthology.orchestra.remote:smithi170:/tmp/tmp.alx1SYOXA1 is 37B 2020-03-19T21:18:00.014 INFO:teuthology.orchestra.run.smithi170:> rm -fr /tmp/tmp.alx1SYOXA1 2020-03-19T21:18:00.027 INFO:teuthology.orchestra.run.smithi170:> mktemp 2020-03-19T21:18:00.080 INFO:teuthology.orchestra.run.smithi170.stdout:/tmp/tmp.ZaBFThSzB6 2020-03-19T21:18:00.081 INFO:teuthology.orchestra.run.smithi170:> sudo cp /var/lib/ceph/osd/ceph-2/fsid /tmp/tmp.ZaBFThSzB6 2020-03-19T21:18:00.148 INFO:teuthology.orchestra.run.smithi170:> sudo chmod 0666 /tmp/tmp.ZaBFThSzB6 2020-03-19T21:18:00.272 DEBUG:teuthology.orchestra.remote:smithi170:/tmp/tmp.ZaBFThSzB6 is 37B 2020-03-19T21:18:00.320 INFO:teuthology.orchestra.run.smithi170:> rm -fr /tmp/tmp.ZaBFThSzB6 2020-03-19T21:18:00.334 INFO:teuthology.orchestra.run.smithi170:> sudo ceph --cluster ceph osd new 32ca3517-1077-4102-9aa4-edfcebe8e3d6 0 2020-03-19T21:18:00.522 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.514+0000 7f9756f4b700 1 Processor -- start 2020-03-19T21:18:00.522 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.515+0000 7f9756f4b700 1 -- start start 2020-03-19T21:18:00.522 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.515+0000 7f9756f4b700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f975008c7d0 0x7f9750087cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T21:18:00.523 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.515+0000 7f9756f4b700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9750090f70 con 0x7f975008bce0 2020-03-19T21:18:00.523 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.515+0000 7f9756f4b700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97500904e0 con 0x7f975008c7d0 2020-03-19T21:18:00.523 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.515+0000 7f9754ce7700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f975008c7d0 0x7f9750087cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:00.523 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.515+0000 7f97554e8700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f975008bce0 0x7f975008c0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:53380/0 (socket says 172.21.15.170:53380) 2020-03-19T21:18:00.523 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.515+0000 7f97554e8700 1 -- 172.21.15.170:0/1849173994 learned_addr learned my addr 172.21.15.170:0/1849173994 (peer_addr_for_me v1:172.21.15.170:0/0) 2020-03-19T21:18:00.523 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.516+0000 7f9754ce7700 1 -- 172.21.15.170:0/1849173994 >> v1:172.21.15.170:6789/0 conn(0x7f975008bce0 legacy=0x7f975008c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:00.523 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.516+0000 7f9747fff700 1 -- 172.21.15.170:0/1849173994 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 272544512 0 0) 0x7f9750090f70 con 0x7f975008bce0 2020-03-19T21:18:00.524 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.516+0000 7f9754ce7700 1 -- 172.21.15.170:0/1849173994 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97500908e0 con 0x7f975008c7d0 2020-03-19T21:18:00.524 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.516+0000 7f9754ce7700 1 --2- 172.21.15.170:0/1849173994 >> v2:172.21.15.170:3300/0 conn(0x7f975008c7d0 0x7f9750087cb0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f974c002f30 tx=0x7f974c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:00.524 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.516+0000 7f9754ce7700 1 --2- 172.21.15.170:0/1849173994 >> v2:172.21.15.170:3300/0 conn(0x7f975008c7d0 0x7f9750087cb0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f974c002f30 tx=0x7f974c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:00.524 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.516+0000 7f9754ce7700 1 --2- 172.21.15.170:0/1849173994 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f975008c7d0 0x7f9750087cb0 secure :-1 s=READY pgs=14 cs=0 l=1 rx=0x7f974c002f30 tx=0x7f974c007a40).ready entity=mon.0 client_cookie=50fe8289a69b33 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:00.524 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.516+0000 7f9754ce7700 1 --2- 172.21.15.170:0/1849173994 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f975008c7d0 0x7f9750087cb0 secure :-1 s=THROTTLE_DONE pgs=14 cs=0 l=1 rx=0x7f974c002f30 tx=0x7f974c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:00.524 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.516+0000 7f9754ce7700 1 --2- 172.21.15.170:0/1849173994 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f975008c7d0 0x7f9750087cb0 secure :-1 s=THROTTLE_DONE pgs=14 cs=0 l=1 rx=0x7f974c002f30 tx=0x7f974c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:00.524 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.516+0000 7f9747fff700 1 -- 172.21.15.170:0/1849173994 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f974c008e80 con 0x7f975008c7d0 2020-03-19T21:18:00.525 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.516+0000 7f9754ce7700 1 --2- 172.21.15.170:0/1849173994 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f975008c7d0 0x7f9750087cb0 secure :-1 s=THROTTLE_DONE pgs=14 cs=0 l=1 rx=0x7f974c002f30 tx=0x7f974c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:00.525 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.516+0000 7f9747fff700 1 -- 172.21.15.170:0/1849173994 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f974c0113f0 con 0x7f975008c7d0 2020-03-19T21:18:00.525 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.516+0000 7f9747fff700 1 -- 172.21.15.170:0/1849173994 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f974c0116d0 con 0x7f975008c7d0 2020-03-19T21:18:00.525 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.516+0000 7f9756f4b700 1 -- 172.21.15.170:0/1849173994 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f975008c7d0 msgr2=0x7f9750087cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:00.525 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.516+0000 7f9756f4b700 1 --2- 172.21.15.170:0/1849173994 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f975008c7d0 0x7f9750087cb0 secure :-1 s=READY pgs=14 cs=0 l=1 rx=0x7f974c002f30 tx=0x7f974c007a40).stop 2020-03-19T21:18:00.525 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.516+0000 7f9756f4b700 1 -- 172.21.15.170:0/1849173994 shutdown_connections 2020-03-19T21:18:00.525 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.516+0000 7f9756f4b700 1 --2- 172.21.15.170:0/1849173994 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f975008c7d0 0x7f9750087cb0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:00.526 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.516+0000 7f9756f4b700 1 -- 172.21.15.170:0/1849173994 >> 172.21.15.170:0/1849173994 conn(0x7f97500951d0 msgr2=0x7f9750091250 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:00.526 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.517+0000 7f9756f4b700 1 -- 172.21.15.170:0/1849173994 shutdown_connections 2020-03-19T21:18:00.526 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.517+0000 7f9756f4b700 1 -- 172.21.15.170:0/1849173994 wait complete. 2020-03-19T21:18:00.526 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.517+0000 7f9756f4b700 1 Processor -- start 2020-03-19T21:18:00.526 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.517+0000 7f9756f4b700 1 -- start start 2020-03-19T21:18:00.526 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.518+0000 7f9756f4b700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f975008bce0 0x7f9750071cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:00.526 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.518+0000 7f9756f4b700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9750075030 con 0x7f975008c7d0 2020-03-19T21:18:00.527 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.518+0000 7f9756f4b700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9750073700 con 0x7f975008bce0 2020-03-19T21:18:00.527 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.518+0000 7f97554e8700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f975008bce0 0x7f9750071cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:00.527 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.518+0000 7f9754ce7700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f975008c7d0 0x7f9750072130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:53384/0 (socket says 172.21.15.170:53384) 2020-03-19T21:18:00.527 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.518+0000 7f9754ce7700 1 -- 172.21.15.170:0/2570466216 learned_addr learned my addr 172.21.15.170:0/2570466216 (peer_addr_for_me v1:172.21.15.170:0/0) 2020-03-19T21:18:00.527 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.518+0000 7f9745ffb700 1 -- 172.21.15.170:0/2570466216 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3742454768 0 0) 0x7f9750075030 con 0x7f975008c7d0 2020-03-19T21:18:00.527 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.518+0000 7f9745ffb700 1 -- 172.21.15.170:0/2570466216 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9730003230 con 0x7f975008c7d0 2020-03-19T21:18:00.527 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.518+0000 7f97554e8700 1 -- 172.21.15.170:0/2570466216 >> v1:172.21.15.170:6789/0 conn(0x7f975008c7d0 legacy=0x7f9750072130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:00.527 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.518+0000 7f97554e8700 1 -- 172.21.15.170:0/2570466216 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9750074660 con 0x7f975008bce0 2020-03-19T21:18:00.528 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.518+0000 7f97554e8700 1 --2- 172.21.15.170:0/2570466216 >> v2:172.21.15.170:3300/0 conn(0x7f975008bce0 0x7f9750071cd0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f9740000b80 tx=0x7f97400108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:00.528 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.519+0000 7f97554e8700 1 --2- 172.21.15.170:0/2570466216 >> v2:172.21.15.170:3300/0 conn(0x7f975008bce0 0x7f9750071cd0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f9740000b80 tx=0x7f97400108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:00.528 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.519+0000 7f97554e8700 1 --2- 172.21.15.170:0/2570466216 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f975008bce0 0x7f9750071cd0 secure :-1 s=READY pgs=16 cs=0 l=1 rx=0x7f9740000b80 tx=0x7f97400108a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:00.528 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.519+0000 7f97554e8700 1 --2- 172.21.15.170:0/2570466216 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f975008bce0 0x7f9750071cd0 secure :-1 s=THROTTLE_DONE pgs=16 cs=0 l=1 rx=0x7f9740000b80 tx=0x7f97400108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:00.528 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.519+0000 7f97554e8700 1 --2- 172.21.15.170:0/2570466216 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f975008bce0 0x7f9750071cd0 secure :-1 s=THROTTLE_DONE pgs=16 cs=0 l=1 rx=0x7f9740000b80 tx=0x7f97400108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:00.528 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.519+0000 7f9745ffb700 1 -- 172.21.15.170:0/2570466216 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f9740005b20 con 0x7f975008bce0 2020-03-19T21:18:00.528 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.519+0000 7f97554e8700 1 --2- 172.21.15.170:0/2570466216 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f975008bce0 0x7f9750071cd0 secure :-1 s=THROTTLE_DONE pgs=16 cs=0 l=1 rx=0x7f9740000b80 tx=0x7f97400108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:00.529 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.519+0000 7f9745ffb700 1 -- 172.21.15.170:0/2570466216 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9740005c80 con 0x7f975008bce0 2020-03-19T21:18:00.529 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.519+0000 7f9756f4b700 1 -- 172.21.15.170:0/2570466216 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9750074900 con 0x7f975008bce0 2020-03-19T21:18:00.529 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.519+0000 7f9745ffb700 1 -- 172.21.15.170:0/2570466216 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f97400060c0 con 0x7f975008bce0 2020-03-19T21:18:00.529 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.519+0000 7f9756f4b700 1 -- 172.21.15.170:0/2570466216 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9750073e40 con 0x7f975008bce0 2020-03-19T21:18:00.529 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.519+0000 7f97554e8700 1 --2- 172.21.15.170:0/2570466216 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f975008bce0 0x7f9750071cd0 secure :-1 s=THROTTLE_DONE pgs=16 cs=0 l=1 rx=0x7f9740000b80 tx=0x7f97400108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:00.529 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.519+0000 7f97554e8700 1 --2- 172.21.15.170:0/2570466216 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f975008bce0 0x7f9750071cd0 secure :-1 s=THROTTLE_DONE pgs=16 cs=0 l=1 rx=0x7f9740000b80 tx=0x7f97400108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:00.529 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.519+0000 7f9745ffb700 1 -- 172.21.15.170:0/2570466216 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 1) v1 ==== 360+0+0 (secure 0 0 0) 0x7f974000a6e0 con 0x7f975008bce0 2020-03-19T21:18:00.530 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.519+0000 7f9745ffb700 1 -- 172.21.15.170:0/2570466216 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 754+0+0 (secure 0 0 0) 0x7f9740009930 con 0x7f975008bce0 2020-03-19T21:18:00.530 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.520+0000 7f9756f4b700 1 -- 172.21.15.170:0/2570466216 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9750074b20 con 0x7f975008bce0 2020-03-19T21:18:00.530 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.522+0000 7f97554e8700 1 --2- 172.21.15.170:0/2570466216 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f975008bce0 0x7f9750071cd0 secure :-1 s=THROTTLE_DONE pgs=16 cs=0 l=1 rx=0x7f9740000b80 tx=0x7f97400108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:00.530 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.522+0000 7f9745ffb700 1 -- 172.21.15.170:0/2570466216 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+68559 (secure 0 0 0) 0x7f97400090b0 con 0x7f975008bce0 2020-03-19T21:18:00.611 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.609+0000 7f9756f4b700 1 -- 172.21.15.170:0/2570466216 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd new", "uuid": "32ca3517-1077-4102-9aa4-edfcebe8e3d6", "id": 0} v 0) v1 -- 0x7f9750056db0 con 0x7f975008bce0 2020-03-19T21:18:00.613 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.611+0000 7f97554e8700 1 --2- 172.21.15.170:0/2570466216 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f975008bce0 0x7f9750071cd0 secure :-1 s=THROTTLE_DONE pgs=16 cs=0 l=1 rx=0x7f9740000b80 tx=0x7f97400108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:00.613 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.611+0000 7f9745ffb700 1 -- 172.21.15.170:0/2570466216 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "32ca3517-1077-4102-9aa4-edfcebe8e3d6", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f9740009c10 con 0x7f975008bce0 2020-03-19T21:18:00.613 INFO:teuthology.orchestra.run.smithi170.stdout:0 2020-03-19T21:18:00.614 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.612+0000 7f9756f4b700 1 -- 172.21.15.170:0/2570466216 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f975008bce0 msgr2=0x7f9750071cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:00.614 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.612+0000 7f9756f4b700 1 --2- 172.21.15.170:0/2570466216 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f975008bce0 0x7f9750071cd0 secure :-1 s=READY pgs=16 cs=0 l=1 rx=0x7f9740000b80 tx=0x7f97400108a0).stop 2020-03-19T21:18:00.615 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.612+0000 7f9756f4b700 1 -- 172.21.15.170:0/2570466216 shutdown_connections 2020-03-19T21:18:00.615 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.612+0000 7f9756f4b700 1 --2- 172.21.15.170:0/2570466216 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f975008bce0 0x7f9750071cd0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:00.615 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.612+0000 7f9756f4b700 1 -- 172.21.15.170:0/2570466216 >> 172.21.15.170:0/2570466216 conn(0x7f97500951d0 msgr2=0x7f9750062870 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:00.615 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.612+0000 7f9756f4b700 1 -- 172.21.15.170:0/2570466216 shutdown_connections 2020-03-19T21:18:00.615 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.612+0000 7f9756f4b700 1 -- 172.21.15.170:0/2570466216 wait complete. 2020-03-19T21:18:00.624 INFO:teuthology.orchestra.run.smithi170:> sudo ceph --cluster ceph osd new 470df64b-60bc-4f5b-9b53-64495cd7c5e6 1 2020-03-19T21:18:00.732 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.728+0000 7f0605030700 1 Processor -- start 2020-03-19T21:18:00.733 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.728+0000 7f0605030700 1 -- start start 2020-03-19T21:18:00.733 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.728+0000 7f0605030700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f060008c7d0 0x7f0600087cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T21:18:00.733 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.728+0000 7f0605030700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0600090f70 con 0x7f060008bce0 2020-03-19T21:18:00.733 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.728+0000 7f0605030700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06000904e0 con 0x7f060008c7d0 2020-03-19T21:18:00.733 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.728+0000 7f05fed9d700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f060008c7d0 0x7f0600087cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:00.733 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.728+0000 7f05ff59e700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f060008bce0 0x7f060008c0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:53388/0 (socket says 172.21.15.170:53388) 2020-03-19T21:18:00.734 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.728+0000 7f05ff59e700 1 -- 172.21.15.170:0/510382015 learned_addr learned my addr 172.21.15.170:0/510382015 (peer_addr_for_me v1:172.21.15.170:0/0) 2020-03-19T21:18:00.734 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.728+0000 7f05fed9d700 1 -- 172.21.15.170:0/510382015 >> v1:172.21.15.170:6789/0 conn(0x7f060008bce0 legacy=0x7f060008c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:00.734 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.728+0000 7f05fed9d700 1 -- 172.21.15.170:0/510382015 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06000908e0 con 0x7f060008c7d0 2020-03-19T21:18:00.734 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.728+0000 7f05fe59c700 1 -- 172.21.15.170:0/510382015 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1811368981 0 0) 0x7f0600090f70 con 0x7f060008bce0 2020-03-19T21:18:00.734 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.728+0000 7f05fed9d700 1 --2- 172.21.15.170:0/510382015 >> v2:172.21.15.170:3300/0 conn(0x7f060008c7d0 0x7f0600087cb0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f05f0002f30 tx=0x7f05f0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:00.734 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.729+0000 7f05fed9d700 1 --2- 172.21.15.170:0/510382015 >> v2:172.21.15.170:3300/0 conn(0x7f060008c7d0 0x7f0600087cb0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f05f0002f30 tx=0x7f05f0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:00.735 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.729+0000 7f05fed9d700 1 --2- 172.21.15.170:0/510382015 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f060008c7d0 0x7f0600087cb0 secure :-1 s=READY pgs=18 cs=0 l=1 rx=0x7f05f0002f30 tx=0x7f05f0007a40).ready entity=mon.0 client_cookie=b1b6acced93ea451 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:00.735 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.729+0000 7f05fed9d700 1 --2- 172.21.15.170:0/510382015 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f060008c7d0 0x7f0600087cb0 secure :-1 s=THROTTLE_DONE pgs=18 cs=0 l=1 rx=0x7f05f0002f30 tx=0x7f05f0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:00.735 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.729+0000 7f05fe59c700 1 -- 172.21.15.170:0/510382015 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f05f0008e80 con 0x7f060008c7d0 2020-03-19T21:18:00.735 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.729+0000 7f05fed9d700 1 --2- 172.21.15.170:0/510382015 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f060008c7d0 0x7f0600087cb0 secure :-1 s=THROTTLE_DONE pgs=18 cs=0 l=1 rx=0x7f05f0002f30 tx=0x7f05f0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:00.735 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.729+0000 7f05fe59c700 1 -- 172.21.15.170:0/510382015 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05f00113f0 con 0x7f060008c7d0 2020-03-19T21:18:00.735 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.729+0000 7f05fed9d700 1 --2- 172.21.15.170:0/510382015 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f060008c7d0 0x7f0600087cb0 secure :-1 s=THROTTLE_DONE pgs=18 cs=0 l=1 rx=0x7f05f0002f30 tx=0x7f05f0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:00.736 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.729+0000 7f05fe59c700 1 -- 172.21.15.170:0/510382015 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f05f00116d0 con 0x7f060008c7d0 2020-03-19T21:18:00.736 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.729+0000 7f0605030700 1 -- 172.21.15.170:0/510382015 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f060008c7d0 msgr2=0x7f0600087cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:00.736 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.729+0000 7f0605030700 1 --2- 172.21.15.170:0/510382015 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f060008c7d0 0x7f0600087cb0 secure :-1 s=READY pgs=18 cs=0 l=1 rx=0x7f05f0002f30 tx=0x7f05f0007a40).stop 2020-03-19T21:18:00.736 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.729+0000 7f0605030700 1 -- 172.21.15.170:0/510382015 shutdown_connections 2020-03-19T21:18:00.736 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.729+0000 7f0605030700 1 --2- 172.21.15.170:0/510382015 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f060008c7d0 0x7f0600087cb0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:00.736 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.729+0000 7f0605030700 1 -- 172.21.15.170:0/510382015 >> 172.21.15.170:0/510382015 conn(0x7f06000951d0 msgr2=0x7f0600091250 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:00.737 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.729+0000 7f0605030700 1 -- 172.21.15.170:0/510382015 shutdown_connections 2020-03-19T21:18:00.737 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.729+0000 7f0605030700 1 -- 172.21.15.170:0/510382015 wait complete. 2020-03-19T21:18:00.737 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.730+0000 7f0605030700 1 Processor -- start 2020-03-19T21:18:00.737 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.730+0000 7f0605030700 1 -- start start 2020-03-19T21:18:00.740 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.731+0000 7f0605030700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f060008c7d0 0x7f06000723e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:00.740 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.731+0000 7f0605030700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0600075030 con 0x7f060008bce0 2020-03-19T21:18:00.740 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.731+0000 7f0605030700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0600073700 con 0x7f060008c7d0 2020-03-19T21:18:00.741 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.731+0000 7f05fed9d700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f060008c7d0 0x7f06000723e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:00.741 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.731+0000 7f05ff59e700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f060008bce0 0x7f0600071cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:53392/0 (socket says 172.21.15.170:53392) 2020-03-19T21:18:00.741 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.731+0000 7f05ff59e700 1 -- 172.21.15.170:0/709898106 learned_addr learned my addr 172.21.15.170:0/709898106 (peer_addr_for_me v1:172.21.15.170:0/0) 2020-03-19T21:18:00.741 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.731+0000 7f05fed9d700 1 -- 172.21.15.170:0/709898106 >> v1:172.21.15.170:6789/0 conn(0x7f060008bce0 legacy=0x7f0600071cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:00.741 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.731+0000 7f05fed9d700 1 -- 172.21.15.170:0/709898106 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0600074660 con 0x7f060008c7d0 2020-03-19T21:18:00.741 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.731+0000 7f05e7fff700 1 -- 172.21.15.170:0/709898106 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4066738357 0 0) 0x7f0600075030 con 0x7f060008bce0 2020-03-19T21:18:00.742 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.731+0000 7f05fed9d700 1 --2- 172.21.15.170:0/709898106 >> v2:172.21.15.170:3300/0 conn(0x7f060008c7d0 0x7f06000723e0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f05f0002f30 tx=0x7f05f0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:00.742 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.731+0000 7f05fed9d700 1 --2- 172.21.15.170:0/709898106 >> v2:172.21.15.170:3300/0 conn(0x7f060008c7d0 0x7f06000723e0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f05f0002f30 tx=0x7f05f0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:00.742 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.731+0000 7f05fed9d700 1 --2- 172.21.15.170:0/709898106 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f060008c7d0 0x7f06000723e0 secure :-1 s=READY pgs=20 cs=0 l=1 rx=0x7f05f0002f30 tx=0x7f05f0007a40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:00.742 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.731+0000 7f05fed9d700 1 --2- 172.21.15.170:0/709898106 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f060008c7d0 0x7f06000723e0 secure :-1 s=THROTTLE_DONE pgs=20 cs=0 l=1 rx=0x7f05f0002f30 tx=0x7f05f0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:00.742 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.731+0000 7f05e7fff700 1 -- 172.21.15.170:0/709898106 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f05f00116c0 con 0x7f060008c7d0 2020-03-19T21:18:00.742 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.731+0000 7f05fed9d700 1 --2- 172.21.15.170:0/709898106 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f060008c7d0 0x7f06000723e0 secure :-1 s=THROTTLE_DONE pgs=20 cs=0 l=1 rx=0x7f05f0002f30 tx=0x7f05f0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:00.742 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.731+0000 7f05fed9d700 1 --2- 172.21.15.170:0/709898106 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f060008c7d0 0x7f06000723e0 secure :-1 s=THROTTLE_DONE pgs=20 cs=0 l=1 rx=0x7f05f0002f30 tx=0x7f05f0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:00.743 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.731+0000 7f05e7fff700 1 -- 172.21.15.170:0/709898106 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05f000f830 con 0x7f060008c7d0 2020-03-19T21:18:00.743 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.731+0000 7f0605030700 1 -- 172.21.15.170:0/709898106 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0600074900 con 0x7f060008c7d0 2020-03-19T21:18:00.743 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.731+0000 7f05e7fff700 1 -- 172.21.15.170:0/709898106 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f05f0007280 con 0x7f060008c7d0 2020-03-19T21:18:00.743 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.731+0000 7f0605030700 1 -- 172.21.15.170:0/709898106 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0600073e40 con 0x7f060008c7d0 2020-03-19T21:18:00.743 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.731+0000 7f05fed9d700 1 --2- 172.21.15.170:0/709898106 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f060008c7d0 0x7f06000723e0 secure :-1 s=THROTTLE_DONE pgs=20 cs=0 l=1 rx=0x7f05f0002f30 tx=0x7f05f0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:00.743 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.732+0000 7f05e7fff700 1 -- 172.21.15.170:0/709898106 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 1) v1 ==== 360+0+0 (secure 0 0 0) 0x7f05f00058a0 con 0x7f060008c7d0 2020-03-19T21:18:00.743 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.732+0000 7f05fed9d700 1 --2- 172.21.15.170:0/709898106 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f060008c7d0 0x7f06000723e0 secure :-1 s=THROTTLE_DONE pgs=20 cs=0 l=1 rx=0x7f05f0002f30 tx=0x7f05f0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:00.744 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.732+0000 7f05e7fff700 1 -- 172.21.15.170:0/709898106 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 869+0+0 (secure 0 0 0) 0x7f05f0005b30 con 0x7f060008c7d0 2020-03-19T21:18:00.744 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.732+0000 7f0605030700 1 -- 172.21.15.170:0/709898106 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0600074b20 con 0x7f060008c7d0 2020-03-19T21:18:00.744 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.734+0000 7f05fed9d700 1 --2- 172.21.15.170:0/709898106 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f060008c7d0 0x7f06000723e0 secure :-1 s=THROTTLE_DONE pgs=20 cs=0 l=1 rx=0x7f05f0002f30 tx=0x7f05f0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:00.744 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.734+0000 7f05e7fff700 1 -- 172.21.15.170:0/709898106 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+68559 (secure 0 0 0) 0x7f05f000bc10 con 0x7f060008c7d0 2020-03-19T21:18:00.826 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.824+0000 7f0605030700 1 -- 172.21.15.170:0/709898106 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd new", "uuid": "470df64b-60bc-4f5b-9b53-64495cd7c5e6", "id": 1} v 0) v1 -- 0x7f0600056db0 con 0x7f060008c7d0 2020-03-19T21:18:00.828 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.826+0000 7f05fed9d700 1 --2- 172.21.15.170:0/709898106 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f060008c7d0 0x7f06000723e0 secure :-1 s=THROTTLE_DONE pgs=20 cs=0 l=1 rx=0x7f05f0002f30 tx=0x7f05f0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:00.828 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.826+0000 7f05e7fff700 1 -- 172.21.15.170:0/709898106 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "470df64b-60bc-4f5b-9b53-64495cd7c5e6", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f05f000bdb0 con 0x7f060008c7d0 2020-03-19T21:18:00.829 INFO:teuthology.orchestra.run.smithi170.stdout:1 2020-03-19T21:18:00.830 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.827+0000 7f0605030700 1 -- 172.21.15.170:0/709898106 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f060008c7d0 msgr2=0x7f06000723e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:00.830 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.827+0000 7f0605030700 1 --2- 172.21.15.170:0/709898106 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f060008c7d0 0x7f06000723e0 secure :-1 s=READY pgs=20 cs=0 l=1 rx=0x7f05f0002f30 tx=0x7f05f0007a40).stop 2020-03-19T21:18:00.830 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.827+0000 7f0605030700 1 -- 172.21.15.170:0/709898106 shutdown_connections 2020-03-19T21:18:00.830 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.827+0000 7f0605030700 1 --2- 172.21.15.170:0/709898106 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f060008c7d0 0x7f06000723e0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:00.831 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.827+0000 7f0605030700 1 -- 172.21.15.170:0/709898106 >> 172.21.15.170:0/709898106 conn(0x7f06000951d0 msgr2=0x7f0600062870 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:00.831 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.827+0000 7f0605030700 1 -- 172.21.15.170:0/709898106 shutdown_connections 2020-03-19T21:18:00.831 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.827+0000 7f0605030700 1 -- 172.21.15.170:0/709898106 wait complete. 2020-03-19T21:18:00.840 INFO:teuthology.orchestra.run.smithi170:> sudo ceph --cluster ceph osd new ccfb9d82-b759-4ff4-8ddf-4f1d741bac5b 2 2020-03-19T21:18:00.950 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.945+0000 7fddeb922700 1 Processor -- start 2020-03-19T21:18:00.950 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.945+0000 7fddeb922700 1 -- start start 2020-03-19T21:18:00.950 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.945+0000 7fddeb922700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fdde408bce0 0x7fdde408c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T21:18:00.950 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.945+0000 7fddeb922700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdde4090f70 con 0x7fdde408c520 2020-03-19T21:18:00.950 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.945+0000 7fddeb922700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdde40904e0 con 0x7fdde408bce0 2020-03-19T21:18:00.951 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.946+0000 7fdde9ebf700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fdde408bce0 0x7fdde408c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:00.951 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.946+0000 7fdde96be700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fdde408c520 0x7fdde4086ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:53396/0 (socket says 172.21.15.170:53396) 2020-03-19T21:18:00.951 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.946+0000 7fdde96be700 1 -- 172.21.15.170:0/1692920680 learned_addr learned my addr 172.21.15.170:0/1692920680 (peer_addr_for_me v1:172.21.15.170:0/0) 2020-03-19T21:18:00.951 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.946+0000 7fdde9ebf700 1 --2- 172.21.15.170:0/1692920680 >> v2:172.21.15.170:3300/0 conn(0x7fdde408bce0 0x7fdde408c0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:52632/0 (socket says 172.21.15.170:52632) 2020-03-19T21:18:00.951 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.946+0000 7fdde9ebf700 1 -- 172.21.15.170:0/1692920680 >> v1:172.21.15.170:6789/0 conn(0x7fdde408c520 legacy=0x7fdde4086ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:00.951 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.946+0000 7fdde8ebd700 1 -- 172.21.15.170:0/1692920680 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3725157093 0 0) 0x7fdde4090f70 con 0x7fdde408c520 2020-03-19T21:18:00.951 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.946+0000 7fdde9ebf700 1 -- 172.21.15.170:0/1692920680 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdde40908e0 con 0x7fdde408bce0 2020-03-19T21:18:00.952 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.946+0000 7fdde9ebf700 1 --2- 172.21.15.170:0/1692920680 >> v2:172.21.15.170:3300/0 conn(0x7fdde408bce0 0x7fdde408c0c0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7fddd8002f30 tx=0x7fddd8007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:00.952 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.946+0000 7fdde9ebf700 1 --2- 172.21.15.170:0/1692920680 >> v2:172.21.15.170:3300/0 conn(0x7fdde408bce0 0x7fdde408c0c0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7fddd8002f30 tx=0x7fddd8007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:00.952 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.946+0000 7fdde9ebf700 1 --2- 172.21.15.170:0/1692920680 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdde408bce0 0x7fdde408c0c0 secure :-1 s=READY pgs=22 cs=0 l=1 rx=0x7fddd8002f30 tx=0x7fddd8007a40).ready entity=mon.0 client_cookie=1780b6b428b38770 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:00.952 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.946+0000 7fdde9ebf700 1 --2- 172.21.15.170:0/1692920680 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdde408bce0 0x7fdde408c0c0 secure :-1 s=THROTTLE_DONE pgs=22 cs=0 l=1 rx=0x7fddd8002f30 tx=0x7fddd8007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:00.952 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.946+0000 7fdde8ebd700 1 -- 172.21.15.170:0/1692920680 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fddd8008e80 con 0x7fdde408bce0 2020-03-19T21:18:00.952 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.946+0000 7fdde9ebf700 1 --2- 172.21.15.170:0/1692920680 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdde408bce0 0x7fdde408c0c0 secure :-1 s=THROTTLE_DONE pgs=22 cs=0 l=1 rx=0x7fddd8002f30 tx=0x7fddd8007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:00.952 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.946+0000 7fdde8ebd700 1 -- 172.21.15.170:0/1692920680 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddd80113f0 con 0x7fdde408bce0 2020-03-19T21:18:00.953 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.946+0000 7fdde9ebf700 1 --2- 172.21.15.170:0/1692920680 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdde408bce0 0x7fdde408c0c0 secure :-1 s=THROTTLE_DONE pgs=22 cs=0 l=1 rx=0x7fddd8002f30 tx=0x7fddd8007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:00.953 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.946+0000 7fddeb922700 1 -- 172.21.15.170:0/1692920680 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdde408bce0 msgr2=0x7fdde408c0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:00.953 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.946+0000 7fdde8ebd700 1 -- 172.21.15.170:0/1692920680 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fddd80116d0 con 0x7fdde408bce0 2020-03-19T21:18:00.953 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.946+0000 7fddeb922700 1 --2- 172.21.15.170:0/1692920680 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdde408bce0 0x7fdde408c0c0 secure :-1 s=READY pgs=22 cs=0 l=1 rx=0x7fddd8002f30 tx=0x7fddd8007a40).stop 2020-03-19T21:18:00.953 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.947+0000 7fddeb922700 1 -- 172.21.15.170:0/1692920680 shutdown_connections 2020-03-19T21:18:00.953 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.947+0000 7fddeb922700 1 --2- 172.21.15.170:0/1692920680 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdde408bce0 0x7fdde408c0c0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:00.953 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.947+0000 7fddeb922700 1 -- 172.21.15.170:0/1692920680 >> 172.21.15.170:0/1692920680 conn(0x7fdde40951d0 msgr2=0x7fdde4091250 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:00.954 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.947+0000 7fddeb922700 1 -- 172.21.15.170:0/1692920680 shutdown_connections 2020-03-19T21:18:00.954 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.947+0000 7fddeb922700 1 -- 172.21.15.170:0/1692920680 wait complete. 2020-03-19T21:18:00.954 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.948+0000 7fddeb922700 1 Processor -- start 2020-03-19T21:18:00.954 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.948+0000 7fddeb922700 1 -- start start 2020-03-19T21:18:00.956 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.948+0000 7fddeb922700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fdde408c520 0x7fdde41268b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:00.957 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.948+0000 7fddeb922700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdde408e9c0 con 0x7fdde408bce0 2020-03-19T21:18:00.957 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.948+0000 7fddeb922700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdde40930f0 con 0x7fdde408c520 2020-03-19T21:18:00.957 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.948+0000 7fdde96be700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fdde408c520 0x7fdde41268b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:00.957 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.948+0000 7fdde9ebf700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fdde408bce0 0x7fdde4085150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:53400/0 (socket says 172.21.15.170:53400) 2020-03-19T21:18:00.957 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.948+0000 7fdde9ebf700 1 -- 172.21.15.170:0/822034489 learned_addr learned my addr 172.21.15.170:0/822034489 (peer_addr_for_me v1:172.21.15.170:0/0) 2020-03-19T21:18:00.957 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.948+0000 7fdde96be700 1 -- 172.21.15.170:0/822034489 >> v1:172.21.15.170:6789/0 conn(0x7fdde408bce0 legacy=0x7fdde4085150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:00.958 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.948+0000 7fddd67fc700 1 -- 172.21.15.170:0/822034489 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 761116326 0 0) 0x7fdde408e9c0 con 0x7fdde408bce0 2020-03-19T21:18:00.958 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.948+0000 7fdde96be700 1 -- 172.21.15.170:0/822034489 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdde4064ca0 con 0x7fdde408c520 2020-03-19T21:18:00.958 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.949+0000 7fdde96be700 1 --2- 172.21.15.170:0/822034489 >> v2:172.21.15.170:3300/0 conn(0x7fdde408c520 0x7fdde41268b0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7fdde0000b80 tx=0x7fdde0010910).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:00.958 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.949+0000 7fdde96be700 1 --2- 172.21.15.170:0/822034489 >> v2:172.21.15.170:3300/0 conn(0x7fdde408c520 0x7fdde41268b0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7fdde0000b80 tx=0x7fdde0010910).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:00.958 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.949+0000 7fdde96be700 1 --2- 172.21.15.170:0/822034489 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdde408c520 0x7fdde41268b0 secure :-1 s=READY pgs=24 cs=0 l=1 rx=0x7fdde0000b80 tx=0x7fdde0010910).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:00.958 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.949+0000 7fdde96be700 1 --2- 172.21.15.170:0/822034489 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdde408c520 0x7fdde41268b0 secure :-1 s=THROTTLE_DONE pgs=24 cs=0 l=1 rx=0x7fdde0000b80 tx=0x7fdde0010910).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:00.958 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.949+0000 7fddd67fc700 1 -- 172.21.15.170:0/822034489 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fdde0005b20 con 0x7fdde408c520 2020-03-19T21:18:00.959 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.949+0000 7fdde96be700 1 --2- 172.21.15.170:0/822034489 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdde408c520 0x7fdde41268b0 secure :-1 s=THROTTLE_DONE pgs=24 cs=0 l=1 rx=0x7fdde0000b80 tx=0x7fdde0010910).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:00.959 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.949+0000 7fdde96be700 1 --2- 172.21.15.170:0/822034489 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdde408c520 0x7fdde41268b0 secure :-1 s=THROTTLE_DONE pgs=24 cs=0 l=1 rx=0x7fdde0000b80 tx=0x7fdde0010910).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:00.959 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.949+0000 7fddd67fc700 1 -- 172.21.15.170:0/822034489 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdde0005c80 con 0x7fdde408c520 2020-03-19T21:18:00.959 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.949+0000 7fddeb922700 1 -- 172.21.15.170:0/822034489 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdde40642d0 con 0x7fdde408c520 2020-03-19T21:18:00.959 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.949+0000 7fddd67fc700 1 -- 172.21.15.170:0/822034489 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fdde00060c0 con 0x7fdde408c520 2020-03-19T21:18:00.959 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.949+0000 7fdde96be700 1 --2- 172.21.15.170:0/822034489 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdde408c520 0x7fdde41268b0 secure :-1 s=THROTTLE_DONE pgs=24 cs=0 l=1 rx=0x7fdde0000b80 tx=0x7fdde0010910).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:00.959 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.949+0000 7fddd67fc700 1 -- 172.21.15.170:0/822034489 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 1) v1 ==== 360+0+0 (secure 0 0 0) 0x7fdde00066f0 con 0x7fdde408c520 2020-03-19T21:18:00.960 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.949+0000 7fddeb922700 1 -- 172.21.15.170:0/822034489 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdde4093910 con 0x7fdde408c520 2020-03-19T21:18:00.960 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.949+0000 7fdde96be700 1 --2- 172.21.15.170:0/822034489 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdde408c520 0x7fdde41268b0 secure :-1 s=THROTTLE_DONE pgs=24 cs=0 l=1 rx=0x7fdde0000b80 tx=0x7fdde0010910).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:00.960 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.949+0000 7fddd67fc700 1 -- 172.21.15.170:0/822034489 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 984+0+0 (secure 0 0 0) 0x7fdde000abc0 con 0x7fdde408c520 2020-03-19T21:18:00.960 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.950+0000 7fddeb922700 1 -- 172.21.15.170:0/822034489 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdde4056db0 con 0x7fdde408c520 2020-03-19T21:18:00.960 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.952+0000 7fdde96be700 1 --2- 172.21.15.170:0/822034489 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdde408c520 0x7fdde41268b0 secure :-1 s=THROTTLE_DONE pgs=24 cs=0 l=1 rx=0x7fdde0000b80 tx=0x7fdde0010910).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:00.961 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:00.952+0000 7fddd67fc700 1 -- 172.21.15.170:0/822034489 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+68559 (secure 0 0 0) 0x7fdde00090b0 con 0x7fdde408c520 2020-03-19T21:18:01.041 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:01.039+0000 7fddeb922700 1 -- 172.21.15.170:0/822034489 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd new", "uuid": "ccfb9d82-b759-4ff4-8ddf-4f1d741bac5b", "id": 2} v 0) v1 -- 0x7fdde40659a0 con 0x7fdde408c520 2020-03-19T21:18:01.047 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:01.045+0000 7fdde96be700 1 --2- 172.21.15.170:0/822034489 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdde408c520 0x7fdde41268b0 secure :-1 s=THROTTLE_DONE pgs=24 cs=0 l=1 rx=0x7fdde0000b80 tx=0x7fdde0010910).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:01.048 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:01.045+0000 7fddd67fc700 1 -- 172.21.15.170:0/822034489 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ccfb9d82-b759-4ff4-8ddf-4f1d741bac5b", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7fdde000ad80 con 0x7fdde408c520 2020-03-19T21:18:01.048 INFO:teuthology.orchestra.run.smithi170.stdout:2 2020-03-19T21:18:01.049 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:01.046+0000 7fddeb922700 1 -- 172.21.15.170:0/822034489 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdde408c520 msgr2=0x7fdde41268b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:01.049 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:01.046+0000 7fddeb922700 1 --2- 172.21.15.170:0/822034489 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdde408c520 0x7fdde41268b0 secure :-1 s=READY pgs=24 cs=0 l=1 rx=0x7fdde0000b80 tx=0x7fdde0010910).stop 2020-03-19T21:18:01.049 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:01.046+0000 7fddeb922700 1 -- 172.21.15.170:0/822034489 shutdown_connections 2020-03-19T21:18:01.049 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:01.046+0000 7fddeb922700 1 --2- 172.21.15.170:0/822034489 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdde408c520 0x7fdde41268b0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:01.050 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:01.046+0000 7fddeb922700 1 -- 172.21.15.170:0/822034489 >> 172.21.15.170:0/822034489 conn(0x7fdde40951d0 msgr2=0x7fdde4087ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:01.050 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:01.046+0000 7fddeb922700 1 -- 172.21.15.170:0/822034489 shutdown_connections 2020-03-19T21:18:01.050 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:01.046+0000 7fddeb922700 1 -- 172.21.15.170:0/822034489 wait complete. 2020-03-19T21:18:01.058 INFO:tasks.ceph.osd.0:Restarting daemon 2020-03-19T21:18:01.059 INFO:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2020-03-19T21:18:01.060 INFO:tasks.ceph.osd.0:Started 2020-03-19T21:18:01.061 INFO:tasks.ceph.osd.1:Restarting daemon 2020-03-19T21:18:01.061 INFO:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2020-03-19T21:18:01.064 INFO:tasks.ceph.osd.1:Started 2020-03-19T21:18:01.064 INFO:tasks.ceph.osd.2:Restarting daemon 2020-03-19T21:18:01.064 INFO:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2020-03-19T21:18:01.066 INFO:tasks.ceph.osd.2:Started 2020-03-19T21:18:01.066 INFO:tasks.ceph:Waiting for OSDs to come up 2020-03-19T21:18:01.368 INFO:teuthology.orchestra.run.smithi170:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2020-03-19T21:18:01.468 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:01.461+0000 7f59475db700 1 Processor -- start 2020-03-19T21:18:01.468 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:01.461+0000 7f59475db700 1 -- start start 2020-03-19T21:18:01.468 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:01.461+0000 7f59475db700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f594008ba00 0x7f594007fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T21:18:01.468 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:01.461+0000 7f59475db700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f594005e8d0 con 0x7f594008af10 2020-03-19T21:18:01.468 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:01.461+0000 7f59475db700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f594005ede0 con 0x7f594008ba00 2020-03-19T21:18:01.468 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:01.461+0000 7f5945377700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f594008ba00 0x7f594007fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:01.469 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:01.461+0000 7f5945b78700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f594008af10 0x7f594008b2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:53416/0 (socket says 172.21.15.170:53416) 2020-03-19T21:18:01.469 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:01.461+0000 7f5945b78700 1 -- 172.21.15.170:0/3930159412 learned_addr learned my addr 172.21.15.170:0/3930159412 (peer_addr_for_me v1:172.21.15.170:0/0) 2020-03-19T21:18:01.469 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:01.462+0000 7f5945377700 1 -- 172.21.15.170:0/3930159412 >> v1:172.21.15.170:6789/0 conn(0x7f594008af10 legacy=0x7f594008b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:01.469 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:01.462+0000 7f5944b76700 1 -- 172.21.15.170:0/3930159412 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1504843597 0 0) 0x7f594005e8d0 con 0x7f594008af10 2020-03-19T21:18:01.469 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:01.462+0000 7f5945377700 1 -- 172.21.15.170:0/3930159412 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5940090af0 con 0x7f594008ba00 2020-03-19T21:18:01.469 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:01.462+0000 7f5945377700 1 --2- 172.21.15.170:0/3930159412 >> v2:172.21.15.170:3300/0 conn(0x7f594008ba00 0x7f594007fc30 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f594008d8a0 tx=0x7f59300088e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:01.470 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:01.462+0000 7f5945377700 1 --2- 172.21.15.170:0/3930159412 >> v2:172.21.15.170:3300/0 conn(0x7f594008ba00 0x7f594007fc30 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f594008d8a0 tx=0x7f59300088e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:01.470 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:01.462+0000 7f5945377700 1 --2- 172.21.15.170:0/3930159412 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f594008ba00 0x7f594007fc30 secure :-1 s=READY pgs=32 cs=0 l=1 rx=0x7f594008d8a0 tx=0x7f59300088e0).ready entity=mon.0 client_cookie=e493f09846e51089 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:01.470 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:01.462+0000 7f5945377700 1 --2- 172.21.15.170:0/3930159412 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f594008ba00 0x7f594007fc30 secure :-1 s=THROTTLE_DONE pgs=32 cs=0 l=1 rx=0x7f594008d8a0 tx=0x7f59300088e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:01.470 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:01.462+0000 7f5944b76700 1 -- 172.21.15.170:0/3930159412 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f59300113f0 con 0x7f594008ba00 2020-03-19T21:18:01.470 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:01.462+0000 7f5945377700 1 --2- 172.21.15.170:0/3930159412 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f594008ba00 0x7f594007fc30 secure :-1 s=THROTTLE_DONE pgs=32 cs=0 l=1 rx=0x7f594008d8a0 tx=0x7f59300088e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:01.470 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:01.462+0000 7f5944b76700 1 -- 172.21.15.170:0/3930159412 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5930011550 con 0x7f594008ba00 2020-03-19T21:18:01.470 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:01.462+0000 7f5945377700 1 --2- 172.21.15.170:0/3930159412 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f594008ba00 0x7f594007fc30 secure :-1 s=THROTTLE_DONE pgs=32 cs=0 l=1 rx=0x7f594008d8a0 tx=0x7f59300088e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:01.471 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:01.462+0000 7f5944b76700 1 -- 172.21.15.170:0/3930159412 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f5930011830 con 0x7f594008ba00 2020-03-19T21:18:01.471 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:01.462+0000 7f59475db700 1 -- 172.21.15.170:0/3930159412 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f594008ba00 msgr2=0x7f594007fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:01.471 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:01.462+0000 7f59475db700 1 --2- 172.21.15.170:0/3930159412 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f594008ba00 0x7f594007fc30 secure :-1 s=READY pgs=32 cs=0 l=1 rx=0x7f594008d8a0 tx=0x7f59300088e0).stop 2020-03-19T21:18:01.471 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:01.463+0000 7f59475db700 1 -- 172.21.15.170:0/3930159412 shutdown_connections 2020-03-19T21:18:01.471 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:01.463+0000 7f59475db700 1 --2- 172.21.15.170:0/3930159412 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f594008ba00 0x7f594007fc30 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:01.471 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:01.463+0000 7f59475db700 1 -- 172.21.15.170:0/3930159412 >> 172.21.15.170:0/3930159412 conn(0x7f59400955a0 msgr2=0x7f5940091460 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:01.471 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:01.463+0000 7f59475db700 1 -- 172.21.15.170:0/3930159412 shutdown_connections 2020-03-19T21:18:01.472 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:01.463+0000 7f59475db700 1 -- 172.21.15.170:0/3930159412 wait complete. 2020-03-19T21:18:01.472 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:01.463+0000 7f59475db700 1 Processor -- start 2020-03-19T21:18:01.472 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:01.464+0000 7f59475db700 1 -- start start 2020-03-19T21:18:01.472 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:01.464+0000 7f59475db700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f594008ba00 0x7f59400871e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:01.472 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:01.464+0000 7f59475db700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59400887d0 con 0x7f594008af10 2020-03-19T21:18:01.472 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:01.464+0000 7f59475db700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59400863d0 con 0x7f594008ba00 2020-03-19T21:18:01.472 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:01.464+0000 7f5945b78700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f594008af10 0x7f5940086ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:53420/0 (socket says 172.21.15.170:53420) 2020-03-19T21:18:01.473 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:01.464+0000 7f5945b78700 1 -- 172.21.15.170:0/1010270366 learned_addr learned my addr 172.21.15.170:0/1010270366 (peer_addr_for_me v1:172.21.15.170:0/0) 2020-03-19T21:18:01.473 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:01.464+0000 7f5945377700 1 --2- 172.21.15.170:0/1010270366 >> v2:172.21.15.170:3300/0 conn(0x7f594008ba00 0x7f59400871e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:01.473 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:01.464+0000 7f59367fc700 1 -- 172.21.15.170:0/1010270366 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1452528366 0 0) 0x7f59400887d0 con 0x7f594008af10 2020-03-19T21:18:01.473 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:01.464+0000 7f59367fc700 1 -- 172.21.15.170:0/1010270366 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5928003230 con 0x7f594008af10 2020-03-19T21:18:01.473 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:01.464+0000 7f5945377700 1 -- 172.21.15.170:0/1010270366 >> v1:172.21.15.170:6789/0 conn(0x7f594008af10 legacy=0x7f5940086ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:01.473 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:01.464+0000 7f5945377700 1 -- 172.21.15.170:0/1010270366 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5940083d00 con 0x7f594008ba00 2020-03-19T21:18:01.473 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:01.464+0000 7f5945377700 1 --2- 172.21.15.170:0/1010270366 >> v2:172.21.15.170:3300/0 conn(0x7f594008ba00 0x7f59400871e0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f594008d8a0 tx=0x7f594008e400).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:01.474 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:01.464+0000 7f5945377700 1 --2- 172.21.15.170:0/1010270366 >> v2:172.21.15.170:3300/0 conn(0x7f594008ba00 0x7f59400871e0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f594008d8a0 tx=0x7f594008e400).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:01.474 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:01.464+0000 7f5945377700 1 --2- 172.21.15.170:0/1010270366 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f594008ba00 0x7f59400871e0 secure :-1 s=READY pgs=34 cs=0 l=1 rx=0x7f594008d8a0 tx=0x7f594008e400).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:01.474 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:01.465+0000 7f5945377700 1 --2- 172.21.15.170:0/1010270366 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f594008ba00 0x7f59400871e0 secure :-1 s=THROTTLE_DONE pgs=34 cs=0 l=1 rx=0x7f594008d8a0 tx=0x7f594008e400).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:01.474 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:01.465+0000 7f59367fc700 1 -- 172.21.15.170:0/1010270366 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f593000f740 con 0x7f594008ba00 2020-03-19T21:18:01.474 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:01.465+0000 7f5945377700 1 --2- 172.21.15.170:0/1010270366 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f594008ba00 0x7f59400871e0 secure :-1 s=THROTTLE_DONE pgs=34 cs=0 l=1 rx=0x7f594008d8a0 tx=0x7f594008e400).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:01.474 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:01.465+0000 7f5945377700 1 --2- 172.21.15.170:0/1010270366 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f594008ba00 0x7f59400871e0 secure :-1 s=THROTTLE_DONE pgs=34 cs=0 l=1 rx=0x7f594008d8a0 tx=0x7f594008e400).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:01.474 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:01.465+0000 7f59367fc700 1 -- 172.21.15.170:0/1010270366 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59300056c0 con 0x7f594008ba00 2020-03-19T21:18:01.475 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:01.465+0000 7f59475db700 1 -- 172.21.15.170:0/1010270366 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5940088a00 con 0x7f594008ba00 2020-03-19T21:18:01.475 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:01.465+0000 7f59367fc700 1 -- 172.21.15.170:0/1010270366 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f59300059a0 con 0x7f594008ba00 2020-03-19T21:18:01.475 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:01.465+0000 7f5945377700 1 --2- 172.21.15.170:0/1010270366 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f594008ba00 0x7f59400871e0 secure :-1 s=THROTTLE_DONE pgs=34 cs=0 l=1 rx=0x7f594008d8a0 tx=0x7f594008e400).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:01.475 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:01.465+0000 7f59475db700 1 -- 172.21.15.170:0/1010270366 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59400884b0 con 0x7f594008ba00 2020-03-19T21:18:01.475 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:01.465+0000 7f59367fc700 1 -- 172.21.15.170:0/1010270366 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 1) v1 ==== 360+0+0 (secure 0 0 0) 0x7f593000a160 con 0x7f594008ba00 2020-03-19T21:18:01.475 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:01.465+0000 7f5945377700 1 --2- 172.21.15.170:0/1010270366 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f594008ba00 0x7f59400871e0 secure :-1 s=THROTTLE_DONE pgs=34 cs=0 l=1 rx=0x7f594008d8a0 tx=0x7f594008e400).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:01.476 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:01.465+0000 7f59367fc700 1 -- 172.21.15.170:0/1010270366 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1099+0+0 (secure 0 0 0) 0x7f593000a3f0 con 0x7f594008ba00 2020-03-19T21:18:01.476 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:01.466+0000 7f59475db700 1 -- 172.21.15.170:0/1010270366 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5940056db0 con 0x7f594008ba00 2020-03-19T21:18:01.476 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:01.468+0000 7f5945377700 1 --2- 172.21.15.170:0/1010270366 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f594008ba00 0x7f59400871e0 secure :-1 s=THROTTLE_DONE pgs=34 cs=0 l=1 rx=0x7f594008d8a0 tx=0x7f594008e400).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:01.478 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:01.468+0000 7f59367fc700 1 -- 172.21.15.170:0/1010270366 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+68559 (secure 0 0 0) 0x7f5940083d00 con 0x7f594008ba00 2020-03-19T21:18:01.544 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:01.541+0000 7f59475db700 1 -- 172.21.15.170:0/1010270366 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f594008a2c0 con 0x7f594008ba00 2020-03-19T21:18:01.546 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:01.542+0000 7f5945377700 1 --2- 172.21.15.170:0/1010270366 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f594008ba00 0x7f59400871e0 secure :-1 s=THROTTLE_DONE pgs=34 cs=0 l=1 rx=0x7f594008d8a0 tx=0x7f594008e400).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:01.546 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:01.542+0000 7f59367fc700 1 -- 172.21.15.170:0/1010270366 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v5) v1 ==== 74+0+3218 (secure 0 0 0) 0x7f593000a5b0 con 0x7f594008ba00 2020-03-19T21:18:01.546 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:01.543+0000 7f59475db700 1 -- 172.21.15.170:0/1010270366 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f594008ba00 msgr2=0x7f59400871e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:01.547 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:01.543+0000 7f59475db700 1 --2- 172.21.15.170:0/1010270366 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f594008ba00 0x7f59400871e0 secure :-1 s=READY pgs=34 cs=0 l=1 rx=0x7f594008d8a0 tx=0x7f594008e400).stop 2020-03-19T21:18:01.547 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:01.543+0000 7f59475db700 1 -- 172.21.15.170:0/1010270366 shutdown_connections 2020-03-19T21:18:01.547 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:01.543+0000 7f59475db700 1 --2- 172.21.15.170:0/1010270366 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f594008ba00 0x7f59400871e0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:01.547 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:01.543+0000 7f59475db700 1 -- 172.21.15.170:0/1010270366 >> 172.21.15.170:0/1010270366 conn(0x7f59400955a0 msgr2=0x7f594007fe50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:01.548 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:01.543+0000 7f59475db700 1 -- 172.21.15.170:0/1010270366 shutdown_connections 2020-03-19T21:18:01.548 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:01.543+0000 7f59475db700 1 -- 172.21.15.170:0/1010270366 wait complete. 2020-03-19T21:18:01.548 INFO:teuthology.misc.health.smithi170.stdout: 2020-03-19T21:18:01.549 INFO:teuthology.misc.health.smithi170.stdout:{"epoch":5,"fsid":"12d0a33b-6b40-4398-90b4-6b5df1da388e","created":"2020-03-19T21:17:59.135747+0000","modified":"2020-03-19T21:18:01.041072+0000","last_up_change":"0.000000","last_in_change":"0.000000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[],"osds":[{"osd":0,"uuid":"32ca3517-1077-4102-9aa4-edfcebe8e3d6","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"470df64b-60bc-4f5b-9b53-64495cd7c5e6","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"ccfb9d82-b759-4ff4-8ddf-4f1d741bac5b","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-03-19T21:18:01.554 DEBUG:teuthology.misc:0 of 3 OSDs are up 2020-03-19T21:18:01.655 INFO:tasks.ceph.osd.0.smithi170.stderr:2020-03-19T21:18:01.652+0000 7faade429ec0 -1 Falling back to public interface 2020-03-19T21:18:01.657 INFO:tasks.ceph.osd.1.smithi170.stderr:2020-03-19T21:18:01.655+0000 7ff72a226ec0 -1 Falling back to public interface 2020-03-19T21:18:01.667 INFO:tasks.ceph.osd.2.smithi170.stderr:2020-03-19T21:18:01.665+0000 7f6d84680ec0 -1 Falling back to public interface 2020-03-19T21:18:02.201 INFO:tasks.ceph.osd.0.smithi170.stderr:2020-03-19T21:18:02.199+0000 7faade429ec0 -1 osd.0 0 log_to_monitors {default=true} 2020-03-19T21:18:02.215 INFO:tasks.ceph.osd.1.smithi170.stderr:2020-03-19T21:18:02.213+0000 7ff72a226ec0 -1 osd.1 0 log_to_monitors {default=true} 2020-03-19T21:18:02.224 INFO:tasks.ceph.osd.2.smithi170.stderr:2020-03-19T21:18:02.222+0000 7f6d84680ec0 -1 osd.2 0 log_to_monitors {default=true} 2020-03-19T21:18:04.162 INFO:tasks.ceph.osd.1.smithi170.stderr:2020-03-19T21:18:04.160+0000 7ff70f1ea700 -1 osd.1 0 waiting for initial osdmap 2020-03-19T21:18:04.163 INFO:tasks.ceph.osd.2.smithi170.stderr:2020-03-19T21:18:04.160+0000 7f6d69644700 -1 osd.2 0 waiting for initial osdmap 2020-03-19T21:18:04.164 INFO:tasks.ceph.osd.0.smithi170.stderr:2020-03-19T21:18:04.162+0000 7faac33ed700 -1 osd.0 0 waiting for initial osdmap 2020-03-19T21:18:07.857 INFO:teuthology.orchestra.run.smithi170:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2020-03-19T21:18:07.963 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:07.958+0000 7f0df1a08700 1 Processor -- start 2020-03-19T21:18:07.963 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:07.958+0000 7f0df1a08700 1 -- start start 2020-03-19T21:18:07.963 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:07.958+0000 7f0df1a08700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f0dec08ba00 0x7f0dec07fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T21:18:07.963 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:07.958+0000 7f0df1a08700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0dec05e8d0 con 0x7f0dec08af10 2020-03-19T21:18:07.963 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:07.959+0000 7f0df1a08700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0dec05ede0 con 0x7f0dec08ba00 2020-03-19T21:18:07.963 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:07.959+0000 7f0deaffd700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f0dec08ba00 0x7f0dec07fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:07.964 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:07.959+0000 7f0deb7fe700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f0dec08af10 0x7f0dec08b2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:53516/0 (socket says 172.21.15.170:53516) 2020-03-19T21:18:07.964 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:07.959+0000 7f0deb7fe700 1 -- 172.21.15.170:0/4238417332 learned_addr learned my addr 172.21.15.170:0/4238417332 (peer_addr_for_me v1:172.21.15.170:0/0) 2020-03-19T21:18:07.964 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:07.959+0000 7f0deaffd700 1 -- 172.21.15.170:0/4238417332 >> v1:172.21.15.170:6789/0 conn(0x7f0dec08af10 legacy=0x7f0dec08b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:07.964 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:07.959+0000 7f0deaffd700 1 -- 172.21.15.170:0/4238417332 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dec090af0 con 0x7f0dec08ba00 2020-03-19T21:18:07.964 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:07.959+0000 7f0dea7fc700 1 -- 172.21.15.170:0/4238417332 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1816023565 0 0) 0x7f0dec05e8d0 con 0x7f0dec08af10 2020-03-19T21:18:07.964 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:07.959+0000 7f0deaffd700 1 --2- 172.21.15.170:0/4238417332 >> v2:172.21.15.170:3300/0 conn(0x7f0dec08ba00 0x7f0dec07fc30 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f0dec08d8a0 tx=0x7f0de00088e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:07.964 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:07.959+0000 7f0deaffd700 1 --2- 172.21.15.170:0/4238417332 >> v2:172.21.15.170:3300/0 conn(0x7f0dec08ba00 0x7f0dec07fc30 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f0dec08d8a0 tx=0x7f0de00088e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:07.965 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:07.959+0000 7f0deaffd700 1 --2- 172.21.15.170:0/4238417332 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0dec08ba00 0x7f0dec07fc30 secure :-1 s=READY pgs=51 cs=0 l=1 rx=0x7f0dec08d8a0 tx=0x7f0de00088e0).ready entity=mon.0 client_cookie=6eb3e7de0fa959f8 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:07.965 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:07.959+0000 7f0deaffd700 1 --2- 172.21.15.170:0/4238417332 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0dec08ba00 0x7f0dec07fc30 secure :-1 s=THROTTLE_DONE pgs=51 cs=0 l=1 rx=0x7f0dec08d8a0 tx=0x7f0de00088e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:07.965 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:07.959+0000 7f0dea7fc700 1 -- 172.21.15.170:0/4238417332 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f0de00113f0 con 0x7f0dec08ba00 2020-03-19T21:18:07.965 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:07.959+0000 7f0deaffd700 1 --2- 172.21.15.170:0/4238417332 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0dec08ba00 0x7f0dec07fc30 secure :-1 s=THROTTLE_DONE pgs=51 cs=0 l=1 rx=0x7f0dec08d8a0 tx=0x7f0de00088e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:07.965 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:07.959+0000 7f0dea7fc700 1 -- 172.21.15.170:0/4238417332 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0de0011550 con 0x7f0dec08ba00 2020-03-19T21:18:07.965 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:07.959+0000 7f0deaffd700 1 --2- 172.21.15.170:0/4238417332 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0dec08ba00 0x7f0dec07fc30 secure :-1 s=THROTTLE_DONE pgs=51 cs=0 l=1 rx=0x7f0dec08d8a0 tx=0x7f0de00088e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:07.965 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:07.959+0000 7f0dea7fc700 1 -- 172.21.15.170:0/4238417332 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f0de0011830 con 0x7f0dec08ba00 2020-03-19T21:18:07.966 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:07.959+0000 7f0df1a08700 1 -- 172.21.15.170:0/4238417332 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0dec08ba00 msgr2=0x7f0dec07fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:07.966 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:07.959+0000 7f0df1a08700 1 --2- 172.21.15.170:0/4238417332 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0dec08ba00 0x7f0dec07fc30 secure :-1 s=READY pgs=51 cs=0 l=1 rx=0x7f0dec08d8a0 tx=0x7f0de00088e0).stop 2020-03-19T21:18:07.966 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:07.960+0000 7f0df1a08700 1 -- 172.21.15.170:0/4238417332 shutdown_connections 2020-03-19T21:18:07.966 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:07.960+0000 7f0df1a08700 1 --2- 172.21.15.170:0/4238417332 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0dec08ba00 0x7f0dec07fc30 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:07.966 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:07.960+0000 7f0df1a08700 1 -- 172.21.15.170:0/4238417332 >> 172.21.15.170:0/4238417332 conn(0x7f0dec0955a0 msgr2=0x7f0dec091460 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:07.966 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:07.960+0000 7f0df1a08700 1 -- 172.21.15.170:0/4238417332 shutdown_connections 2020-03-19T21:18:07.966 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:07.960+0000 7f0df1a08700 1 -- 172.21.15.170:0/4238417332 wait complete. 2020-03-19T21:18:07.967 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:07.961+0000 7f0df1a08700 1 Processor -- start 2020-03-19T21:18:07.970 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:07.961+0000 7f0df1a08700 1 -- start start 2020-03-19T21:18:07.970 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:07.961+0000 7f0df1a08700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f0dec08ba00 0x7f0dec0671c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:07.970 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:07.961+0000 7f0df1a08700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0dec08a880 con 0x7f0dec08af10 2020-03-19T21:18:07.970 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:07.961+0000 7f0df1a08700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0dec08a260 con 0x7f0dec08ba00 2020-03-19T21:18:07.971 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:07.961+0000 7f0deaffd700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f0dec08ba00 0x7f0dec0671c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:07.971 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:07.961+0000 7f0deb7fe700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f0dec08af10 0x7f0dec066ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:53520/0 (socket says 172.21.15.170:53520) 2020-03-19T21:18:07.971 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:07.961+0000 7f0deaffd700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f0dec08ba00 0x7f0dec0671c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:52758/0 (socket says 172.21.15.170:52758) 2020-03-19T21:18:07.971 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:07.961+0000 7f0deb7fe700 1 -- 172.21.15.170:0/3073509896 learned_addr learned my addr 172.21.15.170:0/3073509896 (peer_addr_for_me v1:172.21.15.170:0/0) 2020-03-19T21:18:07.971 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:07.962+0000 7f0deaffd700 1 -- 172.21.15.170:0/3073509896 >> v1:172.21.15.170:6789/0 conn(0x7f0dec08af10 legacy=0x7f0dec066ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:07.971 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:07.962+0000 7f0deaffd700 1 -- 172.21.15.170:0/3073509896 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dec087cb0 con 0x7f0dec08ba00 2020-03-19T21:18:07.971 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:07.962+0000 7f0dd3fff700 1 -- 172.21.15.170:0/3073509896 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2373605554 0 0) 0x7f0dec08a880 con 0x7f0dec08af10 2020-03-19T21:18:07.972 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:07.962+0000 7f0deaffd700 1 --2- 172.21.15.170:0/3073509896 >> v2:172.21.15.170:3300/0 conn(0x7f0dec08ba00 0x7f0dec0671c0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f0dec08d8a0 tx=0x7f0dec08e400).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:07.972 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:07.962+0000 7f0deaffd700 1 --2- 172.21.15.170:0/3073509896 >> v2:172.21.15.170:3300/0 conn(0x7f0dec08ba00 0x7f0dec0671c0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f0dec08d8a0 tx=0x7f0dec08e400).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:07.972 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:07.962+0000 7f0deaffd700 1 --2- 172.21.15.170:0/3073509896 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0dec08ba00 0x7f0dec0671c0 secure :-1 s=READY pgs=53 cs=0 l=1 rx=0x7f0dec08d8a0 tx=0x7f0dec08e400).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:07.972 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:07.962+0000 7f0deaffd700 1 --2- 172.21.15.170:0/3073509896 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0dec08ba00 0x7f0dec0671c0 secure :-1 s=THROTTLE_DONE pgs=53 cs=0 l=1 rx=0x7f0dec08d8a0 tx=0x7f0dec08e400).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:07.972 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:07.962+0000 7f0dd3fff700 1 -- 172.21.15.170:0/3073509896 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f0de000f7c0 con 0x7f0dec08ba00 2020-03-19T21:18:07.972 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:07.962+0000 7f0deaffd700 1 --2- 172.21.15.170:0/3073509896 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0dec08ba00 0x7f0dec0671c0 secure :-1 s=THROTTLE_DONE pgs=53 cs=0 l=1 rx=0x7f0dec08d8a0 tx=0x7f0dec08e400).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:07.972 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:07.962+0000 7f0dd3fff700 1 -- 172.21.15.170:0/3073509896 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0de00056c0 con 0x7f0dec08ba00 2020-03-19T21:18:07.973 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:07.962+0000 7f0deaffd700 1 --2- 172.21.15.170:0/3073509896 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0dec08ba00 0x7f0dec0671c0 secure :-1 s=THROTTLE_DONE pgs=53 cs=0 l=1 rx=0x7f0dec08d8a0 tx=0x7f0dec08e400).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:07.973 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:07.962+0000 7f0df1a08700 1 -- 172.21.15.170:0/3073509896 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0dec0893f0 con 0x7f0dec08ba00 2020-03-19T21:18:07.973 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:07.962+0000 7f0dd3fff700 1 -- 172.21.15.170:0/3073509896 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f0de0005960 con 0x7f0dec08ba00 2020-03-19T21:18:07.973 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:07.962+0000 7f0df1a08700 1 -- 172.21.15.170:0/3073509896 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dec067620 con 0x7f0dec08ba00 2020-03-19T21:18:07.973 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:07.962+0000 7f0deaffd700 1 --2- 172.21.15.170:0/3073509896 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0dec08ba00 0x7f0dec0671c0 secure :-1 s=THROTTLE_DONE pgs=53 cs=0 l=1 rx=0x7f0dec08d8a0 tx=0x7f0dec08e400).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:07.973 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:07.963+0000 7f0dd3fff700 1 -- 172.21.15.170:0/3073509896 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 4) v1 ==== 29603+0+0 (secure 0 0 0) 0x7f0de0005b60 con 0x7f0dec08ba00 2020-03-19T21:18:07.973 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:07.963+0000 7f0deaffd700 1 --2- 172.21.15.170:0/3073509896 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0dec08ba00 0x7f0dec0671c0 secure :-1 s=THROTTLE_DONE pgs=53 cs=0 l=1 rx=0x7f0dec08d8a0 tx=0x7f0dec08e400).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:07.974 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:07.963+0000 7f0dd3fff700 1 --2- 172.21.15.170:0/3073509896 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f0dcc028250 0x7f0dcc02a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:07.974 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:07.963+0000 7f0dd3fff700 1 -- 172.21.15.170:0/3073509896 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 2230+0+0 (secure 0 0 0) 0x7f0de0034630 con 0x7f0dec08ba00 2020-03-19T21:18:07.974 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:07.963+0000 7f0deb7fe700 1 --2- 172.21.15.170:0/3073509896 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f0dcc028250 0x7f0dcc02a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:07.974 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:07.963+0000 7f0df1a08700 1 -- 172.21.15.170:0/3073509896 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0dec08aa10 con 0x7f0dec08ba00 2020-03-19T21:18:07.974 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:07.963+0000 7f0deb7fe700 1 --2- 172.21.15.170:0/3073509896 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f0dcc028250 0x7f0dcc02a710 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f0ddc0058e0 tx=0x7f0ddc001a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:07.974 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:07.963+0000 7f0deb7fe700 1 --2- 172.21.15.170:0/3073509896 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f0dcc028250 0x7f0dcc02a710 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f0ddc0058e0 tx=0x7f0ddc001a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:07.974 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:07.963+0000 7f0deb7fe700 1 --2- 172.21.15.170:0/3073509896 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f0dcc028250 0x7f0dcc02a710 secure :-1 s=READY pgs=13 cs=0 l=1 rx=0x7f0ddc0058e0 tx=0x7f0ddc001a40).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:07.975 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:07.967+0000 7f0deaffd700 1 --2- 172.21.15.170:0/3073509896 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0dec08ba00 0x7f0dec0671c0 secure :-1 s=THROTTLE_DONE pgs=53 cs=0 l=1 rx=0x7f0dec08d8a0 tx=0x7f0dec08e400).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:07.975 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:07.967+0000 7f0dd3fff700 1 -- 172.21.15.170:0/3073509896 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+126772 (secure 0 0 0) 0x7f0de0013290 con 0x7f0dec08ba00 2020-03-19T21:18:08.109 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:08.107+0000 7f0df1a08700 1 -- 172.21.15.170:0/3073509896 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0dec089260 con 0x7f0dec08ba00 2020-03-19T21:18:08.109 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:08.107+0000 7f0deaffd700 1 --2- 172.21.15.170:0/3073509896 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0dec08ba00 0x7f0dec0671c0 secure :-1 s=THROTTLE_DONE pgs=53 cs=0 l=1 rx=0x7f0dec08d8a0 tx=0x7f0dec08e400).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:08.110 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:08.107+0000 7f0dd3fff700 1 -- 172.21.15.170:0/3073509896 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v9) v1 ==== 74+0+4608 (secure 0 0 0) 0x7f0de00130b0 con 0x7f0dec08ba00 2020-03-19T21:18:08.110 INFO:teuthology.misc.health.smithi170.stdout: 2020-03-19T21:18:08.110 INFO:teuthology.misc.health.smithi170.stdout:{"epoch":9,"fsid":"12d0a33b-6b40-4398-90b4-6b5df1da388e","created":"2020-03-19T21:17:59.135747+0000","modified":"2020-03-19T21:18:06.163843+0000","last_up_change":"2020-03-19T21:18:05.158515+0000","last_in_change":"2020-03-19T21:18:05.158515+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[],"osds":[{"osd":0,"uuid":"32ca3517-1077-4102-9aa4-edfcebe8e3d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":20680},{"type":"v1","addr":"172.21.15.170:6801","nonce":20680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":20680},{"type":"v1","addr":"172.21.15.170:6803","nonce":20680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":20680},{"type":"v1","addr":"172.21.15.170:6807","nonce":20680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":20680},{"type":"v1","addr":"172.21.15.170:6805","nonce":20680}]},"public_addr":"172.21.15.170:6801/20680","cluster_addr":"172.21.15.170:6803/20680","heartbeat_back_addr":"172.21.15.170:6807/20680","heartbeat_front_addr":"172.21.15.170:6805/20680","state":["exists","up"]},{"osd":1,"uuid":"470df64b-60bc-4f5b-9b53-64495cd7c5e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":20681},{"type":"v1","addr":"172.21.15.170:6809","nonce":20681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":20681},{"type":"v1","addr":"172.21.15.170:6811","nonce":20681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":20681},{"type":"v1","addr":"172.21.15.170:6815","nonce":20681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":20681},{"type":"v1","addr":"172.21.15.170:6813","nonce":20681}]},"public_addr":"172.21.15.170:6809/20681","cluster_addr":"172.21.15.170:6811/20681","heartbeat_back_addr":"172.21.15.170:6815/20681","heartbeat_front_addr":"172.21.15.170:6813/20681","state":["exists","up"]},{"osd":2,"uuid":"ccfb9d82-b759-4ff4-8ddf-4f1d741bac5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":20682},{"type":"v1","addr":"172.21.15.170:6817","nonce":20682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":20682},{"type":"v1","addr":"172.21.15.170:6819","nonce":20682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":20682},{"type":"v1","addr":"172.21.15.170:6823","nonce":20682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":20682},{"type":"v1","addr":"172.21.15.170:6821","nonce":20682}]},"public_addr":"172.21.15.170:6817/20682","cluster_addr":"172.21.15.170:6819/20682","heartbeat_back_addr":"172.21.15.170:6823/20682","heartbeat_front_addr":"172.21.15.170:6821/20682","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2020-03-19T21:18:03.177293+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2020-03-19T21:18:03.263179+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2020-03-19T21:18:03.183223+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-03-19T21:18:08.111 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:08.108+0000 7f0df1a08700 1 -- 172.21.15.170:0/3073509896 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f0dcc028250 msgr2=0x7f0dcc02a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:08.111 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:08.108+0000 7f0df1a08700 1 --2- 172.21.15.170:0/3073509896 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f0dcc028250 0x7f0dcc02a710 secure :-1 s=READY pgs=13 cs=0 l=1 rx=0x7f0ddc0058e0 tx=0x7f0ddc001a40).stop 2020-03-19T21:18:08.111 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:08.108+0000 7f0df1a08700 1 -- 172.21.15.170:0/3073509896 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0dec08ba00 msgr2=0x7f0dec0671c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:08.112 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:08.108+0000 7f0df1a08700 1 --2- 172.21.15.170:0/3073509896 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0dec08ba00 0x7f0dec0671c0 secure :-1 s=READY pgs=53 cs=0 l=1 rx=0x7f0dec08d8a0 tx=0x7f0dec08e400).stop 2020-03-19T21:18:08.112 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:08.108+0000 7f0df1a08700 1 -- 172.21.15.170:0/3073509896 shutdown_connections 2020-03-19T21:18:08.112 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:08.108+0000 7f0df1a08700 1 --2- 172.21.15.170:0/3073509896 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f0dcc028250 0x7f0dcc02a710 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:08.112 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:08.108+0000 7f0df1a08700 1 --2- 172.21.15.170:0/3073509896 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0dec08ba00 0x7f0dec0671c0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:08.112 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:08.108+0000 7f0df1a08700 1 -- 172.21.15.170:0/3073509896 >> 172.21.15.170:0/3073509896 conn(0x7f0dec0955a0 msgr2=0x7f0dec07fe50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:08.112 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:08.108+0000 7f0df1a08700 1 -- 172.21.15.170:0/3073509896 shutdown_connections 2020-03-19T21:18:08.112 INFO:teuthology.misc.health.smithi170.stderr:2020-03-19T21:18:08.108+0000 7f0df1a08700 1 -- 172.21.15.170:0/3073509896 wait complete. 2020-03-19T21:18:08.120 DEBUG:teuthology.misc:3 of 3 OSDs are up 2020-03-19T21:18:08.120 INFO:tasks.ceph:Creating RBD pool 2020-03-19T21:18:08.120 INFO:teuthology.orchestra.run.smithi170:> sudo ceph --cluster ceph osd pool create rbd 8 2020-03-19T21:18:08.227 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:08.223+0000 7f7a03df0700 1 Processor -- start 2020-03-19T21:18:08.228 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:08.223+0000 7f7a03df0700 1 -- start start 2020-03-19T21:18:08.228 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:08.223+0000 7f7a03df0700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f79fc08f010 0x7f79fc08f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T21:18:08.228 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:08.223+0000 7f7a03df0700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79fc0902c0 con 0x7f79fc08f850 2020-03-19T21:18:08.228 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:08.223+0000 7f7a03df0700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79fc0904f0 con 0x7f79fc08f010 2020-03-19T21:18:08.228 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:08.224+0000 7f7a0238d700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f79fc08f010 0x7f79fc08f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:08.228 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:08.224+0000 7f7a01b8c700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f79fc08f850 0x7f79fc136ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:53528/0 (socket says 172.21.15.170:53528) 2020-03-19T21:18:08.229 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:08.224+0000 7f7a01b8c700 1 -- 172.21.15.170:0/1894943886 learned_addr learned my addr 172.21.15.170:0/1894943886 (peer_addr_for_me v1:172.21.15.170:0/0) 2020-03-19T21:18:08.229 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:08.224+0000 7f7a0238d700 1 -- 172.21.15.170:0/1894943886 >> v1:172.21.15.170:6789/0 conn(0x7f79fc08f850 legacy=0x7f79fc136ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:08.229 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:08.224+0000 7f7a0238d700 1 -- 172.21.15.170:0/1894943886 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79fc0954e0 con 0x7f79fc08f010 2020-03-19T21:18:08.229 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:08.224+0000 7f7a0138b700 1 -- 172.21.15.170:0/1894943886 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1391120597 0 0) 0x7f79fc0902c0 con 0x7f79fc08f850 2020-03-19T21:18:08.229 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:08.224+0000 7f7a0238d700 1 --2- 172.21.15.170:0/1894943886 >> v2:172.21.15.170:3300/0 conn(0x7f79fc08f010 0x7f79fc08f3f0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f79f8002f30 tx=0x7f79f8007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:08.229 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:08.224+0000 7f7a0238d700 1 --2- 172.21.15.170:0/1894943886 >> v2:172.21.15.170:3300/0 conn(0x7f79fc08f010 0x7f79fc08f3f0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f79f8002f30 tx=0x7f79f8007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:08.230 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:08.224+0000 7f7a0238d700 1 --2- 172.21.15.170:0/1894943886 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f79fc08f010 0x7f79fc08f3f0 secure :-1 s=READY pgs=55 cs=0 l=1 rx=0x7f79f8002f30 tx=0x7f79f8007a40).ready entity=mon.0 client_cookie=749c1946961021bc server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:08.230 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:08.224+0000 7f7a0238d700 1 --2- 172.21.15.170:0/1894943886 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f79fc08f010 0x7f79fc08f3f0 secure :-1 s=THROTTLE_DONE pgs=55 cs=0 l=1 rx=0x7f79f8002f30 tx=0x7f79f8007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:08.230 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:08.224+0000 7f7a0138b700 1 -- 172.21.15.170:0/1894943886 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f79f8008e80 con 0x7f79fc08f010 2020-03-19T21:18:08.230 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:08.224+0000 7f7a0238d700 1 --2- 172.21.15.170:0/1894943886 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f79fc08f010 0x7f79fc08f3f0 secure :-1 s=THROTTLE_DONE pgs=55 cs=0 l=1 rx=0x7f79f8002f30 tx=0x7f79f8007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:08.230 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:08.224+0000 7f7a0138b700 1 -- 172.21.15.170:0/1894943886 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79f80113f0 con 0x7f79fc08f010 2020-03-19T21:18:08.230 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:08.224+0000 7f7a0238d700 1 --2- 172.21.15.170:0/1894943886 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f79fc08f010 0x7f79fc08f3f0 secure :-1 s=THROTTLE_DONE pgs=55 cs=0 l=1 rx=0x7f79f8002f30 tx=0x7f79f8007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:08.230 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:08.224+0000 7f7a0138b700 1 -- 172.21.15.170:0/1894943886 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f79f80116d0 con 0x7f79fc08f010 2020-03-19T21:18:08.231 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:08.224+0000 7f7a03df0700 1 -- 172.21.15.170:0/1894943886 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f79fc08f010 msgr2=0x7f79fc08f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:08.231 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:08.224+0000 7f7a03df0700 1 --2- 172.21.15.170:0/1894943886 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f79fc08f010 0x7f79fc08f3f0 secure :-1 s=READY pgs=55 cs=0 l=1 rx=0x7f79f8002f30 tx=0x7f79f8007a40).stop 2020-03-19T21:18:08.231 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:08.225+0000 7f7a03df0700 1 -- 172.21.15.170:0/1894943886 shutdown_connections 2020-03-19T21:18:08.231 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:08.225+0000 7f7a03df0700 1 --2- 172.21.15.170:0/1894943886 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f79fc08f010 0x7f79fc08f3f0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:08.231 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:08.225+0000 7f7a03df0700 1 -- 172.21.15.170:0/1894943886 >> 172.21.15.170:0/1894943886 conn(0x7f79fc095090 msgr2=0x7f79fc091110 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:08.231 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:08.225+0000 7f7a03df0700 1 -- 172.21.15.170:0/1894943886 shutdown_connections 2020-03-19T21:18:08.231 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:08.225+0000 7f7a03df0700 1 -- 172.21.15.170:0/1894943886 wait complete. 2020-03-19T21:18:08.231 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:08.226+0000 7f7a03df0700 1 Processor -- start 2020-03-19T21:18:08.232 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:08.226+0000 7f7a03df0700 1 -- start start 2020-03-19T21:18:08.235 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:08.226+0000 7f7a03df0700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f79fc08f010 0x7f79fc092d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:08.235 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:08.226+0000 7f7a03df0700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79fc07d500 con 0x7f79fc08f850 2020-03-19T21:18:08.235 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:08.226+0000 7f7a03df0700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79fc07d7a0 con 0x7f79fc08f010 2020-03-19T21:18:08.235 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:08.226+0000 7f7a0238d700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f79fc08f010 0x7f79fc092d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:08.235 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:08.226+0000 7f7a01b8c700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f79fc08f850 0x7f79fc072780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:53532/0 (socket says 172.21.15.170:53532) 2020-03-19T21:18:08.235 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:08.226+0000 7f7a01b8c700 1 -- 172.21.15.170:0/4271143931 learned_addr learned my addr 172.21.15.170:0/4271143931 (peer_addr_for_me v1:172.21.15.170:0/0) 2020-03-19T21:18:08.235 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:08.226+0000 7f7a0238d700 1 -- 172.21.15.170:0/4271143931 >> v1:172.21.15.170:6789/0 conn(0x7f79fc08f850 legacy=0x7f79fc072780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:08.236 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:08.226+0000 7f7a0238d700 1 -- 172.21.15.170:0/4271143931 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79fc074b80 con 0x7f79fc08f010 2020-03-19T21:18:08.236 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:08.226+0000 7f79f2ffd700 1 -- 172.21.15.170:0/4271143931 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 93618912 0 0) 0x7f79fc07d500 con 0x7f79fc08f850 2020-03-19T21:18:08.236 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:08.226+0000 7f7a0238d700 1 --2- 172.21.15.170:0/4271143931 >> v2:172.21.15.170:3300/0 conn(0x7f79fc08f010 0x7f79fc092d10 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f79fc073ae0 tx=0x7f79f8007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:08.236 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:08.226+0000 7f7a0238d700 1 --2- 172.21.15.170:0/4271143931 >> v2:172.21.15.170:3300/0 conn(0x7f79fc08f010 0x7f79fc092d10 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f79fc073ae0 tx=0x7f79f8007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:08.236 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:08.226+0000 7f7a0238d700 1 --2- 172.21.15.170:0/4271143931 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f79fc08f010 0x7f79fc092d10 secure :-1 s=READY pgs=57 cs=0 l=1 rx=0x7f79fc073ae0 tx=0x7f79f8007a40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:08.236 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:08.227+0000 7f7a0238d700 1 --2- 172.21.15.170:0/4271143931 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f79fc08f010 0x7f79fc092d10 secure :-1 s=THROTTLE_DONE pgs=57 cs=0 l=1 rx=0x7f79fc073ae0 tx=0x7f79f8007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:08.236 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:08.227+0000 7f79f2ffd700 1 -- 172.21.15.170:0/4271143931 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f79f8011d80 con 0x7f79fc08f010 2020-03-19T21:18:08.237 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:08.227+0000 7f7a0238d700 1 --2- 172.21.15.170:0/4271143931 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f79fc08f010 0x7f79fc092d10 secure :-1 s=THROTTLE_DONE pgs=57 cs=0 l=1 rx=0x7f79fc073ae0 tx=0x7f79f8007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:08.237 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:08.227+0000 7f7a0238d700 1 --2- 172.21.15.170:0/4271143931 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f79fc08f010 0x7f79fc092d10 secure :-1 s=THROTTLE_DONE pgs=57 cs=0 l=1 rx=0x7f79fc073ae0 tx=0x7f79f8007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:08.237 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:08.227+0000 7f79f2ffd700 1 -- 172.21.15.170:0/4271143931 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79f8007150 con 0x7f79fc08f010 2020-03-19T21:18:08.237 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:08.227+0000 7f7a03df0700 1 -- 172.21.15.170:0/4271143931 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79fc093930 con 0x7f79fc08f010 2020-03-19T21:18:08.237 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:08.227+0000 7f79f2ffd700 1 -- 172.21.15.170:0/4271143931 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f79f80073f0 con 0x7f79fc08f010 2020-03-19T21:18:08.237 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:08.227+0000 7f7a03df0700 1 -- 172.21.15.170:0/4271143931 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79fc073ff0 con 0x7f79fc08f010 2020-03-19T21:18:08.237 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:08.227+0000 7f7a0238d700 1 --2- 172.21.15.170:0/4271143931 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f79fc08f010 0x7f79fc092d10 secure :-1 s=THROTTLE_DONE pgs=57 cs=0 l=1 rx=0x7f79fc073ae0 tx=0x7f79f8007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:08.238 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:08.227+0000 7f79f2ffd700 1 -- 172.21.15.170:0/4271143931 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 4) v1 ==== 29603+0+0 (secure 0 0 0) 0x7f79f80056c0 con 0x7f79fc08f010 2020-03-19T21:18:08.238 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:08.227+0000 7f7a0238d700 1 --2- 172.21.15.170:0/4271143931 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f79fc08f010 0x7f79fc092d10 secure :-1 s=THROTTLE_DONE pgs=57 cs=0 l=1 rx=0x7f79fc073ae0 tx=0x7f79f8007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:08.238 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:08.227+0000 7f79f2ffd700 1 --2- 172.21.15.170:0/4271143931 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f79e4028200 0x7f79e402a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:08.238 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:08.227+0000 7f79f2ffd700 1 -- 172.21.15.170:0/4271143931 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 2230+0+0 (secure 0 0 0) 0x7f79f8034880 con 0x7f79fc08f010 2020-03-19T21:18:08.238 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:08.227+0000 7f7a01b8c700 1 --2- 172.21.15.170:0/4271143931 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f79e4028200 0x7f79e402a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:08.238 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:08.228+0000 7f7a01b8c700 1 --2- 172.21.15.170:0/4271143931 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f79e4028200 0x7f79e402a6c0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f79ec005500 tx=0x7f79ec0016f0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:08.238 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:08.228+0000 7f7a01b8c700 1 --2- 172.21.15.170:0/4271143931 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f79e4028200 0x7f79e402a6c0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f79ec005500 tx=0x7f79ec0016f0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:08.239 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:08.228+0000 7f7a01b8c700 1 --2- 172.21.15.170:0/4271143931 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f79e4028200 0x7f79e402a6c0 secure :-1 s=READY pgs=14 cs=0 l=1 rx=0x7f79ec005500 tx=0x7f79ec0016f0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:08.239 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:08.228+0000 7f7a03df0700 1 -- 172.21.15.170:0/4271143931 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79fc056db0 con 0x7f79fc08f010 2020-03-19T21:18:08.239 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:08.231+0000 7f7a0238d700 1 --2- 172.21.15.170:0/4271143931 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f79fc08f010 0x7f79fc092d10 secure :-1 s=THROTTLE_DONE pgs=57 cs=0 l=1 rx=0x7f79fc073ae0 tx=0x7f79f8007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:08.239 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:08.231+0000 7f79f2ffd700 1 -- 172.21.15.170:0/4271143931 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+126772 (secure 0 0 0) 0x7f79f8006070 con 0x7f79fc08f010 2020-03-19T21:18:08.371 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:08.369+0000 7f7a03df0700 1 -- 172.21.15.170:0/4271143931 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7f79fc074910 con 0x7f79fc08f010 2020-03-19T21:18:09.174 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:09.172+0000 7f7a0238d700 1 --2- 172.21.15.170:0/4271143931 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f79fc08f010 0x7f79fc092d10 secure :-1 s=THROTTLE_DONE pgs=57 cs=0 l=1 rx=0x7f79fc073ae0 tx=0x7f79f8007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:09.174 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:09.172+0000 7f79f2ffd700 1 -- 172.21.15.170:0/4271143931 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v10) v1 ==== 109+0+0 (secure 0 0 0) 0x7f79f8009de0 con 0x7f79fc08f010 2020-03-19T21:18:09.178 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:09.176+0000 7f7a03df0700 1 -- 172.21.15.170:0/4271143931 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f79e4028200 msgr2=0x7f79e402a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:09.178 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:09.176+0000 7f7a03df0700 1 --2- 172.21.15.170:0/4271143931 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f79e4028200 0x7f79e402a6c0 secure :-1 s=READY pgs=14 cs=0 l=1 rx=0x7f79ec005500 tx=0x7f79ec0016f0).stop 2020-03-19T21:18:09.179 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:09.176+0000 7f7a03df0700 1 -- 172.21.15.170:0/4271143931 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f79fc08f010 msgr2=0x7f79fc092d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:09.179 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:09.176+0000 7f7a03df0700 1 --2- 172.21.15.170:0/4271143931 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f79fc08f010 0x7f79fc092d10 secure :-1 s=READY pgs=57 cs=0 l=1 rx=0x7f79fc073ae0 tx=0x7f79f8007a40).stop 2020-03-19T21:18:09.179 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:09.176+0000 7f7a03df0700 1 -- 172.21.15.170:0/4271143931 shutdown_connections 2020-03-19T21:18:09.179 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:09.176+0000 7f7a03df0700 1 --2- 172.21.15.170:0/4271143931 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f79e4028200 0x7f79e402a6c0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:09.179 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:09.176+0000 7f7a03df0700 1 --2- 172.21.15.170:0/4271143931 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f79fc08f010 0x7f79fc092d10 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:09.179 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:09.176+0000 7f7a03df0700 1 -- 172.21.15.170:0/4271143931 >> 172.21.15.170:0/4271143931 conn(0x7f79fc095090 msgr2=0x7f79fc061310 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:09.179 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:09.176+0000 7f7a03df0700 1 -- 172.21.15.170:0/4271143931 shutdown_connections 2020-03-19T21:18:09.180 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:09.177+0000 7f7a03df0700 1 -- 172.21.15.170:0/4271143931 wait complete. 2020-03-19T21:18:09.180 INFO:teuthology.orchestra.run.smithi170.stderr:pool 'rbd' created 2020-03-19T21:18:09.191 INFO:teuthology.orchestra.run.smithi170:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2020-03-19T21:18:09.305 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:09.301+0000 7ff0f45c9700 1 Processor -- start 2020-03-19T21:18:09.306 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:09.301+0000 7ff0f45c9700 1 -- start start 2020-03-19T21:18:09.306 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:09.301+0000 7ff0f45c9700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7ff0ec08f790 0x7ff0ec137d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T21:18:09.306 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:09.301+0000 7ff0f45c9700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0ec0902a0 con 0x7ff0ec091890 2020-03-19T21:18:09.306 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:09.301+0000 7ff0f45c9700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0ec0600e0 con 0x7ff0ec08f790 2020-03-19T21:18:09.306 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:09.301+0000 7ff0f2365700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7ff0ec08f790 0x7ff0ec137d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:09.306 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:09.301+0000 7ff0f2b66700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7ff0ec091890 0x7ff0ec08f080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:53536/0 (socket says 172.21.15.170:53536) 2020-03-19T21:18:09.307 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:09.301+0000 7ff0f2b66700 1 -- 172.21.15.170:0/3007075262 learned_addr learned my addr 172.21.15.170:0/3007075262 (peer_addr_for_me v1:172.21.15.170:0/0) 2020-03-19T21:18:09.307 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:09.302+0000 7ff0f2365700 1 -- 172.21.15.170:0/3007075262 >> v1:172.21.15.170:6789/0 conn(0x7ff0ec091890 legacy=0x7ff0ec08f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:09.307 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:09.302+0000 7ff0f2365700 1 -- 172.21.15.170:0/3007075262 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0ec0954e0 con 0x7ff0ec08f790 2020-03-19T21:18:09.307 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:09.302+0000 7ff0f1b64700 1 -- 172.21.15.170:0/3007075262 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 449896199 0 0) 0x7ff0ec0902a0 con 0x7ff0ec091890 2020-03-19T21:18:09.307 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:09.302+0000 7ff0f2365700 1 --2- 172.21.15.170:0/3007075262 >> v2:172.21.15.170:3300/0 conn(0x7ff0ec08f790 0x7ff0ec137d60 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7ff0e0002f30 tx=0x7ff0e0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:09.307 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:09.302+0000 7ff0f2365700 1 --2- 172.21.15.170:0/3007075262 >> v2:172.21.15.170:3300/0 conn(0x7ff0ec08f790 0x7ff0ec137d60 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7ff0e0002f30 tx=0x7ff0e0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:09.307 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:09.302+0000 7ff0f2365700 1 --2- 172.21.15.170:0/3007075262 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff0ec08f790 0x7ff0ec137d60 secure :-1 s=READY pgs=59 cs=0 l=1 rx=0x7ff0e0002f30 tx=0x7ff0e0007a40).ready entity=mon.0 client_cookie=80f94adad95fc27f server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:09.308 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:09.302+0000 7ff0f2365700 1 --2- 172.21.15.170:0/3007075262 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff0ec08f790 0x7ff0ec137d60 secure :-1 s=THROTTLE_DONE pgs=59 cs=0 l=1 rx=0x7ff0e0002f30 tx=0x7ff0e0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:09.308 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:09.302+0000 7ff0f1b64700 1 -- 172.21.15.170:0/3007075262 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7ff0e0008e80 con 0x7ff0ec08f790 2020-03-19T21:18:09.308 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:09.302+0000 7ff0f2365700 1 --2- 172.21.15.170:0/3007075262 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff0ec08f790 0x7ff0ec137d60 secure :-1 s=THROTTLE_DONE pgs=59 cs=0 l=1 rx=0x7ff0e0002f30 tx=0x7ff0e0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:09.308 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:09.302+0000 7ff0f1b64700 1 -- 172.21.15.170:0/3007075262 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0e00113f0 con 0x7ff0ec08f790 2020-03-19T21:18:09.308 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:09.302+0000 7ff0f2365700 1 --2- 172.21.15.170:0/3007075262 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff0ec08f790 0x7ff0ec137d60 secure :-1 s=THROTTLE_DONE pgs=59 cs=0 l=1 rx=0x7ff0e0002f30 tx=0x7ff0e0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:09.308 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:09.302+0000 7ff0f1b64700 1 -- 172.21.15.170:0/3007075262 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7ff0e00116d0 con 0x7ff0ec08f790 2020-03-19T21:18:09.308 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:09.302+0000 7ff0f45c9700 1 -- 172.21.15.170:0/3007075262 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff0ec08f790 msgr2=0x7ff0ec137d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:09.309 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:09.302+0000 7ff0f45c9700 1 --2- 172.21.15.170:0/3007075262 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff0ec08f790 0x7ff0ec137d60 secure :-1 s=READY pgs=59 cs=0 l=1 rx=0x7ff0e0002f30 tx=0x7ff0e0007a40).stop 2020-03-19T21:18:09.309 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:09.302+0000 7ff0f45c9700 1 -- 172.21.15.170:0/3007075262 shutdown_connections 2020-03-19T21:18:09.309 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:09.302+0000 7ff0f45c9700 1 --2- 172.21.15.170:0/3007075262 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff0ec08f790 0x7ff0ec137d60 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:09.309 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:09.303+0000 7ff0f45c9700 1 -- 172.21.15.170:0/3007075262 >> 172.21.15.170:0/3007075262 conn(0x7ff0ec095100 msgr2=0x7ff0ec091180 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:09.309 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:09.303+0000 7ff0f45c9700 1 -- 172.21.15.170:0/3007075262 shutdown_connections 2020-03-19T21:18:09.309 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:09.303+0000 7ff0f45c9700 1 -- 172.21.15.170:0/3007075262 wait complete. 2020-03-19T21:18:09.309 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:09.303+0000 7ff0f45c9700 1 Processor -- start 2020-03-19T21:18:09.310 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:09.303+0000 7ff0f45c9700 1 -- start start 2020-03-19T21:18:09.312 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:09.304+0000 7ff0f45c9700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7ff0ec091890 0x7ff0ec07d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:09.312 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:09.304+0000 7ff0f45c9700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0ec075740 con 0x7ff0ec08f790 2020-03-19T21:18:09.313 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:09.304+0000 7ff0f45c9700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0ec074ab0 con 0x7ff0ec091890 2020-03-19T21:18:09.313 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:09.304+0000 7ff0f2365700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7ff0ec091890 0x7ff0ec07d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:09.313 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:09.304+0000 7ff0f2b66700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7ff0ec08f790 0x7ff0ec0729d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:53538/0 (socket says 172.21.15.170:53538) 2020-03-19T21:18:09.313 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:09.304+0000 7ff0f2b66700 1 -- 172.21.15.170:0/4120992567 learned_addr learned my addr 172.21.15.170:0/4120992567 (peer_addr_for_me v1:172.21.15.170:0/0) 2020-03-19T21:18:09.313 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:09.304+0000 7ff0f2365700 1 -- 172.21.15.170:0/4120992567 >> v1:172.21.15.170:6789/0 conn(0x7ff0ec08f790 legacy=0x7ff0ec0729d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:09.313 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:09.304+0000 7ff0f2365700 1 -- 172.21.15.170:0/4120992567 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0ec075430 con 0x7ff0ec091890 2020-03-19T21:18:09.314 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:09.304+0000 7ff0eb7fe700 1 -- 172.21.15.170:0/4120992567 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4236826957 0 0) 0x7ff0ec075740 con 0x7ff0ec08f790 2020-03-19T21:18:09.314 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:09.304+0000 7ff0f2365700 1 --2- 172.21.15.170:0/4120992567 >> v2:172.21.15.170:3300/0 conn(0x7ff0ec091890 0x7ff0ec07d730 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7ff0e0002f30 tx=0x7ff0e0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:09.314 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:09.304+0000 7ff0f2365700 1 --2- 172.21.15.170:0/4120992567 >> v2:172.21.15.170:3300/0 conn(0x7ff0ec091890 0x7ff0ec07d730 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7ff0e0002f30 tx=0x7ff0e0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:09.314 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:09.304+0000 7ff0f2365700 1 --2- 172.21.15.170:0/4120992567 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff0ec091890 0x7ff0ec07d730 secure :-1 s=READY pgs=61 cs=0 l=1 rx=0x7ff0e0002f30 tx=0x7ff0e0007a40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:09.314 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:09.304+0000 7ff0f2365700 1 --2- 172.21.15.170:0/4120992567 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff0ec091890 0x7ff0ec07d730 secure :-1 s=THROTTLE_DONE pgs=61 cs=0 l=1 rx=0x7ff0e0002f30 tx=0x7ff0e0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:09.314 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:09.304+0000 7ff0eb7fe700 1 -- 172.21.15.170:0/4120992567 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7ff0e0007150 con 0x7ff0ec091890 2020-03-19T21:18:09.314 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:09.304+0000 7ff0f2365700 1 --2- 172.21.15.170:0/4120992567 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff0ec091890 0x7ff0ec07d730 secure :-1 s=THROTTLE_DONE pgs=61 cs=0 l=1 rx=0x7ff0e0002f30 tx=0x7ff0e0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:09.314 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:09.304+0000 7ff0f2365700 1 --2- 172.21.15.170:0/4120992567 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff0ec091890 0x7ff0ec07d730 secure :-1 s=THROTTLE_DONE pgs=61 cs=0 l=1 rx=0x7ff0e0002f30 tx=0x7ff0e0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:09.315 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:09.304+0000 7ff0eb7fe700 1 -- 172.21.15.170:0/4120992567 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0e00072b0 con 0x7ff0ec091890 2020-03-19T21:18:09.315 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:09.304+0000 7ff0f45c9700 1 -- 172.21.15.170:0/4120992567 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0ec0931c0 con 0x7ff0ec091890 2020-03-19T21:18:09.315 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:09.304+0000 7ff0eb7fe700 1 -- 172.21.15.170:0/4120992567 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7ff0e0007590 con 0x7ff0ec091890 2020-03-19T21:18:09.315 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:09.304+0000 7ff0f45c9700 1 -- 172.21.15.170:0/4120992567 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0ec074340 con 0x7ff0ec091890 2020-03-19T21:18:09.315 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:09.305+0000 7ff0f2365700 1 --2- 172.21.15.170:0/4120992567 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff0ec091890 0x7ff0ec07d730 secure :-1 s=THROTTLE_DONE pgs=61 cs=0 l=1 rx=0x7ff0e0002f30 tx=0x7ff0e0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:09.315 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:09.305+0000 7ff0eb7fe700 1 -- 172.21.15.170:0/4120992567 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 4) v1 ==== 29603+0+0 (secure 0 0 0) 0x7ff0e00056c0 con 0x7ff0ec091890 2020-03-19T21:18:09.315 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:09.305+0000 7ff0f2365700 1 --2- 172.21.15.170:0/4120992567 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff0ec091890 0x7ff0ec07d730 secure :-1 s=THROTTLE_DONE pgs=61 cs=0 l=1 rx=0x7ff0e0002f30 tx=0x7ff0e0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:09.316 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:09.305+0000 7ff0eb7fe700 1 --2- 172.21.15.170:0/4120992567 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7ff0cc028230 0x7ff0cc02a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:09.316 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:09.305+0000 7ff0eb7fe700 1 -- 172.21.15.170:0/4120992567 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 2572+0+0 (secure 0 0 0) 0x7ff0e00348e0 con 0x7ff0ec091890 2020-03-19T21:18:09.316 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:09.305+0000 7ff0f2b66700 1 --2- 172.21.15.170:0/4120992567 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7ff0cc028230 0x7ff0cc02a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:09.316 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:09.305+0000 7ff0f2b66700 1 --2- 172.21.15.170:0/4120992567 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7ff0cc028230 0x7ff0cc02a6f0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7ff0dc005500 tx=0x7ff0dc001700).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:09.316 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:09.306+0000 7ff0f2b66700 1 --2- 172.21.15.170:0/4120992567 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7ff0cc028230 0x7ff0cc02a6f0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7ff0dc005500 tx=0x7ff0dc001700).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:09.316 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:09.306+0000 7ff0f2b66700 1 --2- 172.21.15.170:0/4120992567 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7ff0cc028230 0x7ff0cc02a6f0 secure :-1 s=READY pgs=15 cs=0 l=1 rx=0x7ff0dc005500 tx=0x7ff0dc001700).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:09.316 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:09.306+0000 7ff0f45c9700 1 -- 172.21.15.170:0/4120992567 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0ec056db0 con 0x7ff0ec091890 2020-03-19T21:18:09.316 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:09.309+0000 7ff0f2365700 1 --2- 172.21.15.170:0/4120992567 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff0ec091890 0x7ff0ec07d730 secure :-1 s=THROTTLE_DONE pgs=61 cs=0 l=1 rx=0x7ff0e0002f30 tx=0x7ff0e0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:09.317 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:09.309+0000 7ff0eb7fe700 1 -- 172.21.15.170:0/4120992567 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+126772 (secure 0 0 0) 0x7ff0e0006250 con 0x7ff0ec091890 2020-03-19T21:18:09.449 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:09.447+0000 7ff0f45c9700 1 -- 172.21.15.170:0/4120992567 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7ff0ec091d20 con 0x7ff0ec091890 2020-03-19T21:18:10.178 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.175+0000 7ff0f2365700 1 --2- 172.21.15.170:0/4120992567 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff0ec091890 0x7ff0ec07d730 secure :-1 s=THROTTLE_DONE pgs=61 cs=0 l=1 rx=0x7ff0e0002f30 tx=0x7ff0e0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:10.178 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.175+0000 7ff0eb7fe700 1 -- 172.21.15.170:0/4120992567 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v11) v1 ==== 173+0+0 (secure 0 0 0) 0x7ff0e0006070 con 0x7ff0ec091890 2020-03-19T21:18:10.181 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.178+0000 7ff0f45c9700 1 -- 172.21.15.170:0/4120992567 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7ff0cc028230 msgr2=0x7ff0cc02a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:10.181 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.178+0000 7ff0f45c9700 1 --2- 172.21.15.170:0/4120992567 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7ff0cc028230 0x7ff0cc02a6f0 secure :-1 s=READY pgs=15 cs=0 l=1 rx=0x7ff0dc005500 tx=0x7ff0dc001700).stop 2020-03-19T21:18:10.181 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.178+0000 7ff0f45c9700 1 -- 172.21.15.170:0/4120992567 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff0ec091890 msgr2=0x7ff0ec07d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:10.181 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.178+0000 7ff0f45c9700 1 --2- 172.21.15.170:0/4120992567 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff0ec091890 0x7ff0ec07d730 secure :-1 s=READY pgs=61 cs=0 l=1 rx=0x7ff0e0002f30 tx=0x7ff0e0007a40).stop 2020-03-19T21:18:10.181 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.178+0000 7ff0f45c9700 1 -- 172.21.15.170:0/4120992567 shutdown_connections 2020-03-19T21:18:10.181 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.178+0000 7ff0f45c9700 1 --2- 172.21.15.170:0/4120992567 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7ff0cc028230 0x7ff0cc02a6f0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:10.182 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.178+0000 7ff0f45c9700 1 --2- 172.21.15.170:0/4120992567 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff0ec091890 0x7ff0ec07d730 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:10.182 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.178+0000 7ff0f45c9700 1 -- 172.21.15.170:0/4120992567 >> 172.21.15.170:0/4120992567 conn(0x7ff0ec095100 msgr2=0x7ff0ec0632b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:10.182 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.178+0000 7ff0f45c9700 1 -- 172.21.15.170:0/4120992567 shutdown_connections 2020-03-19T21:18:10.182 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.179+0000 7ff0f45c9700 1 -- 172.21.15.170:0/4120992567 wait complete. 2020-03-19T21:18:10.183 INFO:teuthology.orchestra.run.smithi170.stderr:enabled application 'rbd' on pool 'rbd' 2020-03-19T21:18:10.196 INFO:tasks.ceph:Setting up CephFS filesystem... 2020-03-19T21:18:10.196 INFO:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-03-19T21:18:10.307 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.305+0000 7f4cf70f8700 1 Processor -- start 2020-03-19T21:18:10.308 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.305+0000 7f4cf70f8700 1 -- start start 2020-03-19T21:18:10.312 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.306+0000 7f4cf70f8700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f4cf0066ed0 0x7f4cf0122900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T21:18:10.312 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.306+0000 7f4cf70f8700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4cf0097860 con 0x7f4cf0122d60 2020-03-19T21:18:10.313 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.306+0000 7f4cf70f8700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cf0122400 con 0x7f4cf0066ed0 2020-03-19T21:18:10.313 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.306+0000 7f4cf4e94700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f4cf0066ed0 0x7f4cf0122900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:10.313 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.306+0000 7f4ceffff700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f4cf0122d60 0x7f4cf0061180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:53546/0 (socket says 172.21.15.170:53546) 2020-03-19T21:18:10.313 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.306+0000 7f4ceffff700 1 -- 172.21.15.170:0/3584688575 learned_addr learned my addr 172.21.15.170:0/3584688575 (peer_addr_for_me v1:172.21.15.170:0/0) 2020-03-19T21:18:10.313 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.306+0000 7f4cf4e94700 1 -- 172.21.15.170:0/3584688575 >> v1:172.21.15.170:6789/0 conn(0x7f4cf0122d60 legacy=0x7f4cf0061180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:10.313 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.306+0000 7f4cf4e94700 1 -- 172.21.15.170:0/3584688575 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cf00927d0 con 0x7f4cf0066ed0 2020-03-19T21:18:10.314 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.306+0000 7f4cf4e94700 1 --2- 172.21.15.170:0/3584688575 >> v2:172.21.15.170:3300/0 conn(0x7f4cf0066ed0 0x7f4cf0122900 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f4ce0002f30 tx=0x7f4ce0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:10.314 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.306+0000 7f4cf4e94700 1 --2- 172.21.15.170:0/3584688575 >> v2:172.21.15.170:3300/0 conn(0x7f4cf0066ed0 0x7f4cf0122900 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f4ce0002f30 tx=0x7f4ce0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:10.315 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.306+0000 7f4cf4e94700 1 --2- 172.21.15.170:0/3584688575 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4cf0066ed0 0x7f4cf0122900 secure :-1 s=READY pgs=63 cs=0 l=1 rx=0x7f4ce0002f30 tx=0x7f4ce0007a40).ready entity=mon.0 client_cookie=c071779130ff18cd server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:10.315 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.306+0000 7f4cf4e94700 1 --2- 172.21.15.170:0/3584688575 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4cf0066ed0 0x7f4cf0122900 secure :-1 s=THROTTLE_DONE pgs=63 cs=0 l=1 rx=0x7f4ce0002f30 tx=0x7f4ce0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:10.315 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.306+0000 7f4cf4e94700 1 --2- 172.21.15.170:0/3584688575 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4cf0066ed0 0x7f4cf0122900 secure :-1 s=THROTTLE_DONE pgs=63 cs=0 l=1 rx=0x7f4ce0002f30 tx=0x7f4ce0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:10.315 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.306+0000 7f4cef7fe700 1 -- 172.21.15.170:0/3584688575 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f4ce0008e80 con 0x7f4cf0066ed0 2020-03-19T21:18:10.315 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.306+0000 7f4cf4e94700 1 --2- 172.21.15.170:0/3584688575 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4cf0066ed0 0x7f4cf0122900 secure :-1 s=THROTTLE_DONE pgs=63 cs=0 l=1 rx=0x7f4ce0002f30 tx=0x7f4ce0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:10.315 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.306+0000 7f4cef7fe700 1 -- 172.21.15.170:0/3584688575 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ce00113f0 con 0x7f4cf0066ed0 2020-03-19T21:18:10.316 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.306+0000 7f4cef7fe700 1 -- 172.21.15.170:0/3584688575 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f4ce0011710 con 0x7f4cf0066ed0 2020-03-19T21:18:10.316 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.306+0000 7f4cf70f8700 1 -- 172.21.15.170:0/3584688575 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4cf0066ed0 msgr2=0x7f4cf0122900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:10.316 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.306+0000 7f4cf70f8700 1 --2- 172.21.15.170:0/3584688575 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4cf0066ed0 0x7f4cf0122900 secure :-1 s=READY pgs=63 cs=0 l=1 rx=0x7f4ce0002f30 tx=0x7f4ce0007a40).stop 2020-03-19T21:18:10.316 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.307+0000 7f4cf70f8700 1 -- 172.21.15.170:0/3584688575 shutdown_connections 2020-03-19T21:18:10.316 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.307+0000 7f4cf70f8700 1 --2- 172.21.15.170:0/3584688575 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4cf0066ed0 0x7f4cf0122900 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:10.316 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.307+0000 7f4cf70f8700 1 -- 172.21.15.170:0/3584688575 >> 172.21.15.170:0/3584688575 conn(0x7f4cf0063390 msgr2=0x7f4cf00657b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:10.316 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.307+0000 7f4cf70f8700 1 -- 172.21.15.170:0/3584688575 shutdown_connections 2020-03-19T21:18:10.317 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.307+0000 7f4cf70f8700 1 -- 172.21.15.170:0/3584688575 wait complete. 2020-03-19T21:18:10.317 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.308+0000 7f4cf70f8700 1 Processor -- start 2020-03-19T21:18:10.317 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.308+0000 7f4cf70f8700 1 -- start start 2020-03-19T21:18:10.317 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.308+0000 7f4cf70f8700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f4cf0122d60 0x7f4cf011f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:10.317 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.308+0000 7f4cf70f8700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4cf010f950 con 0x7f4cf0066ed0 2020-03-19T21:18:10.317 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.308+0000 7f4cf70f8700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cf010ddc0 con 0x7f4cf0122d60 2020-03-19T21:18:10.317 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.308+0000 7f4ceffff700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f4cf0122d60 0x7f4cf011f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:10.318 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.308+0000 7f4cf4e94700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f4cf0066ed0 0x7f4cf011ec40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:53548/0 (socket says 172.21.15.170:53548) 2020-03-19T21:18:10.318 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.308+0000 7f4cf4e94700 1 -- 172.21.15.170:0/3467038626 learned_addr learned my addr 172.21.15.170:0/3467038626 (peer_addr_for_me v1:172.21.15.170:0/0) 2020-03-19T21:18:10.318 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.308+0000 7f4ceffff700 1 -- 172.21.15.170:0/3467038626 >> v1:172.21.15.170:6789/0 conn(0x7f4cf0066ed0 legacy=0x7f4cf011ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:10.318 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.308+0000 7f4ceffff700 1 -- 172.21.15.170:0/3467038626 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cf010e5f0 con 0x7f4cf0122d60 2020-03-19T21:18:10.318 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.309+0000 7f4ceffff700 1 --2- 172.21.15.170:0/3467038626 >> v2:172.21.15.170:3300/0 conn(0x7f4cf0122d60 0x7f4cf011f350 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f4cf010d600 tx=0x7f4ce40108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:10.318 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.309+0000 7f4ceffff700 1 --2- 172.21.15.170:0/3467038626 >> v2:172.21.15.170:3300/0 conn(0x7f4cf0122d60 0x7f4cf011f350 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f4cf010d600 tx=0x7f4ce40108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:10.318 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.309+0000 7f4ceffff700 1 --2- 172.21.15.170:0/3467038626 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4cf0122d60 0x7f4cf011f350 secure :-1 s=READY pgs=65 cs=0 l=1 rx=0x7f4cf010d600 tx=0x7f4ce40108a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:10.319 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.309+0000 7f4ceffff700 1 --2- 172.21.15.170:0/3467038626 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4cf0122d60 0x7f4cf011f350 secure :-1 s=THROTTLE_DONE pgs=65 cs=0 l=1 rx=0x7f4cf010d600 tx=0x7f4ce40108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:10.319 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.309+0000 7f4cedffb700 1 -- 172.21.15.170:0/3467038626 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f4ce4005b20 con 0x7f4cf0122d60 2020-03-19T21:18:10.319 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.309+0000 7f4ceffff700 1 --2- 172.21.15.170:0/3467038626 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4cf0122d60 0x7f4cf011f350 secure :-1 s=THROTTLE_DONE pgs=65 cs=0 l=1 rx=0x7f4cf010d600 tx=0x7f4ce40108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:10.319 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.309+0000 7f4cf70f8700 1 -- 172.21.15.170:0/3467038626 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cf010dab0 con 0x7f4cf0122d60 2020-03-19T21:18:10.319 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.309+0000 7f4cedffb700 1 -- 172.21.15.170:0/3467038626 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ce4005c80 con 0x7f4cf0122d60 2020-03-19T21:18:10.319 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.309+0000 7f4ceffff700 1 --2- 172.21.15.170:0/3467038626 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4cf0122d60 0x7f4cf011f350 secure :-1 s=THROTTLE_DONE pgs=65 cs=0 l=1 rx=0x7f4cf010d600 tx=0x7f4ce40108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:10.319 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.309+0000 7f4cedffb700 1 -- 172.21.15.170:0/3467038626 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f4ce40060c0 con 0x7f4cf0122d60 2020-03-19T21:18:10.319 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.309+0000 7f4cf70f8700 1 -- 172.21.15.170:0/3467038626 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cf010d810 con 0x7f4cf0122d60 2020-03-19T21:18:10.320 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.309+0000 7f4ceffff700 1 --2- 172.21.15.170:0/3467038626 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4cf0122d60 0x7f4cf011f350 secure :-1 s=THROTTLE_DONE pgs=65 cs=0 l=1 rx=0x7f4cf010d600 tx=0x7f4ce40108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:10.320 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.309+0000 7f4cedffb700 1 -- 172.21.15.170:0/3467038626 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 4) v1 ==== 29603+0+0 (secure 0 0 0) 0x7f4ce400a320 con 0x7f4cf0122d60 2020-03-19T21:18:10.320 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.309+0000 7f4ceffff700 1 --2- 172.21.15.170:0/3467038626 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4cf0122d60 0x7f4cf011f350 secure :-1 s=THROTTLE_DONE pgs=65 cs=0 l=1 rx=0x7f4cf010d600 tx=0x7f4ce40108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:10.320 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.309+0000 7f4cedffb700 1 --2- 172.21.15.170:0/3467038626 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f4cd0028320 0x7f4cd002a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:10.320 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.309+0000 7f4cedffb700 1 -- 172.21.15.170:0/3467038626 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(11..11 src has 1..11) v4 ==== 2583+0+0 (secure 0 0 0) 0x7f4ce4033fc0 con 0x7f4cf0122d60 2020-03-19T21:18:10.320 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.309+0000 7f4cf4e94700 1 --2- 172.21.15.170:0/3467038626 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f4cd0028320 0x7f4cd002a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:10.320 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.310+0000 7f4cf4e94700 1 --2- 172.21.15.170:0/3467038626 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f4cd0028320 0x7f4cd002a7e0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f4cf005bf00 tx=0x7f4ce0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:10.321 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.310+0000 7f4cf4e94700 1 --2- 172.21.15.170:0/3467038626 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f4cd0028320 0x7f4cd002a7e0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f4cf005bf00 tx=0x7f4ce0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:10.321 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.310+0000 7f4cf4e94700 1 --2- 172.21.15.170:0/3467038626 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f4cd0028320 0x7f4cd002a7e0 secure :-1 s=READY pgs=16 cs=0 l=1 rx=0x7f4cf005bf00 tx=0x7f4ce0007a40).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:10.321 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.310+0000 7f4cf70f8700 1 -- 172.21.15.170:0/3467038626 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cf0097860 con 0x7f4cf0122d60 2020-03-19T21:18:10.321 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.313+0000 7f4ceffff700 1 --2- 172.21.15.170:0/3467038626 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4cf0122d60 0x7f4cf011f350 secure :-1 s=THROTTLE_DONE pgs=65 cs=0 l=1 rx=0x7f4cf010d600 tx=0x7f4ce40108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:10.321 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.313+0000 7f4cedffb700 1 -- 172.21.15.170:0/3467038626 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+126772 (secure 0 0 0) 0x7f4cf0097860 con 0x7f4cf0122d60 2020-03-19T21:18:10.457 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.455+0000 7f4cf70f8700 1 -- 172.21.15.170:0/3467038626 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4cf0097860 con 0x7f4cf0122d60 2020-03-19T21:18:10.458 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.456+0000 7f4ceffff700 1 --2- 172.21.15.170:0/3467038626 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4cf0122d60 0x7f4cf011f350 secure :-1 s=THROTTLE_DONE pgs=65 cs=0 l=1 rx=0x7f4cf010d600 tx=0x7f4ce40108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:10.458 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.456+0000 7f4cedffb700 1 -- 172.21.15.170:0/3467038626 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v11) v1 ==== 74+0+5993 (secure 0 0 0) 0x7f4cf0097860 con 0x7f4cf0122d60 2020-03-19T21:18:10.458 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:18:10.459 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":11,"fsid":"12d0a33b-6b40-4398-90b4-6b5df1da388e","created":"2020-03-19T21:17:59.135747+0000","modified":"2020-03-19T21:18:10.172038+0000","last_up_change":"2020-03-19T21:18:05.158515+0000","last_in_change":"2020-03-19T21:18:05.158515+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":3,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2020-03-19T21:18:08.391604+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"32ca3517-1077-4102-9aa4-edfcebe8e3d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":20680},{"type":"v1","addr":"172.21.15.170:6801","nonce":20680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":20680},{"type":"v1","addr":"172.21.15.170:6803","nonce":20680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":20680},{"type":"v1","addr":"172.21.15.170:6807","nonce":20680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":20680},{"type":"v1","addr":"172.21.15.170:6805","nonce":20680}]},"public_addr":"172.21.15.170:6801/20680","cluster_addr":"172.21.15.170:6803/20680","heartbeat_back_addr":"172.21.15.170:6807/20680","heartbeat_front_addr":"172.21.15.170:6805/20680","state":["exists","up"]},{"osd":1,"uuid":"470df64b-60bc-4f5b-9b53-64495cd7c5e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":20681},{"type":"v1","addr":"172.21.15.170:6809","nonce":20681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":20681},{"type":"v1","addr":"172.21.15.170:6811","nonce":20681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":20681},{"type":"v1","addr":"172.21.15.170:6815","nonce":20681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":20681},{"type":"v1","addr":"172.21.15.170:6813","nonce":20681}]},"public_addr":"172.21.15.170:6809/20681","cluster_addr":"172.21.15.170:6811/20681","heartbeat_back_addr":"172.21.15.170:6815/20681","heartbeat_front_addr":"172.21.15.170:6813/20681","state":["exists","up"]},{"osd":2,"uuid":"ccfb9d82-b759-4ff4-8ddf-4f1d741bac5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":20682},{"type":"v1","addr":"172.21.15.170:6817","nonce":20682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":20682},{"type":"v1","addr":"172.21.15.170:6819","nonce":20682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":20682},{"type":"v1","addr":"172.21.15.170:6823","nonce":20682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":20682},{"type":"v1","addr":"172.21.15.170:6821","nonce":20682}]},"public_addr":"172.21.15.170:6817/20682","cluster_addr":"172.21.15.170:6819/20682","heartbeat_back_addr":"172.21.15.170:6823/20682","heartbeat_front_addr":"172.21.15.170:6821/20682","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2020-03-19T21:18:03.177293+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2020-03-19T21:18:03.263179+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2020-03-19T21:18:03.183223+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-03-19T21:18:10.460 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.457+0000 7f4cf70f8700 1 -- 172.21.15.170:0/3467038626 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f4cd0028320 msgr2=0x7f4cd002a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:10.460 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.457+0000 7f4cf70f8700 1 --2- 172.21.15.170:0/3467038626 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f4cd0028320 0x7f4cd002a7e0 secure :-1 s=READY pgs=16 cs=0 l=1 rx=0x7f4cf005bf00 tx=0x7f4ce0007a40).stop 2020-03-19T21:18:10.460 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.457+0000 7f4cf70f8700 1 -- 172.21.15.170:0/3467038626 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4cf0122d60 msgr2=0x7f4cf011f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:10.460 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.457+0000 7f4cf70f8700 1 --2- 172.21.15.170:0/3467038626 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4cf0122d60 0x7f4cf011f350 secure :-1 s=READY pgs=65 cs=0 l=1 rx=0x7f4cf010d600 tx=0x7f4ce40108a0).stop 2020-03-19T21:18:10.461 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.457+0000 7f4cf70f8700 1 -- 172.21.15.170:0/3467038626 shutdown_connections 2020-03-19T21:18:10.461 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.457+0000 7f4cf70f8700 1 --2- 172.21.15.170:0/3467038626 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f4cd0028320 0x7f4cd002a7e0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:10.461 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.457+0000 7f4cf70f8700 1 --2- 172.21.15.170:0/3467038626 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4cf0122d60 0x7f4cf011f350 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:10.461 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.457+0000 7f4cf70f8700 1 -- 172.21.15.170:0/3467038626 >> 172.21.15.170:0/3467038626 conn(0x7f4cf0063390 msgr2=0x7f4cf01218d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:10.461 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.457+0000 7f4cf70f8700 1 -- 172.21.15.170:0/3467038626 shutdown_connections 2020-03-19T21:18:10.461 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.457+0000 7f4cf70f8700 1 -- 172.21.15.170:0/3467038626 wait complete. 2020-03-19T21:18:10.470 INFO:tasks.cephfs.filesystem.ceph_manager:[{u'cache_target_full_ratio_micro': 800000, u'fast_read': False, u'stripe_width': 0, u'flags_names': u'hashpspool,creating', u'tier_of': -1, u'hit_set_grade_decay_rate': 0, u'pg_placement_num': 8, u'use_gmt_hitset': True, u'cache_target_dirty_ratio_micro': 400000, u'create_time': u'2020-03-19T21:18:08.391604+0000', u'snap_epoch': 0, u'erasure_code_profile': u'', u'pg_autoscale_mode': u'off', u'snap_seq': 0, u'expected_num_objects': 0, u'size': 2, u'pg_num_pending': 8, u'auid': 0, u'cache_min_flush_age': 0, u'hit_set_period': 0, u'min_read_recency_for_promote': 0, u'target_max_objects': 0, u'pg_placement_num_target': 8, u'pg_num': 8, u'type': 1, u'grade_table': [], u'pool_name': u'rbd', u'cache_min_evict_age': 0, u'snap_mode': u'selfmanaged', u'pg_num_target': 8, u'cache_mode': u'none', u'min_size': 1, u'cache_target_dirty_high_ratio_micro': 600000, u'object_hash': 2, u'last_pg_merge_meta': {u'ready_epoch': 0, u'source_version': u"0'0", u'source_pgid': u'0.0', u'last_epoch_clean': 0, u'target_version': u"0'0", u'last_epoch_started': 0}, u'write_tier': -1, u'last_force_op_resend_preluminous': u'0', u'pool': 1, u'removed_snaps': u'[]', u'crush_rule': 0, u'tiers': [], u'hit_set_params': {u'type': u'none'}, u'last_force_op_resend': u'0', u'pool_snaps': [], u'quota_max_objects': 0, u'last_force_op_resend_prenautilus': u'0', u'application_metadata': {u'rbd': {}}, u'options': {}, u'hit_set_count': 0, u'flags': 32769, u'target_max_bytes': 0, u'quota_max_bytes': 0, u'hit_set_search_last_n': 0, u'last_change': u'11', u'min_write_recency_for_promote': 0, u'read_tier': -1}] 2020-03-19T21:18:10.470 INFO:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-03-19T21:18:10.584 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.582+0000 7f810ef37700 1 Processor -- start 2020-03-19T21:18:10.584 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.582+0000 7f810ef37700 1 -- start start 2020-03-19T21:18:10.589 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.582+0000 7f810ef37700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f81081224c0 0x7f8108122ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T21:18:10.589 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.582+0000 7f810ef37700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81081228a0 con 0x7f8108122f40 2020-03-19T21:18:10.589 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.582+0000 7f810ef37700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8108092200 con 0x7f81081224c0 2020-03-19T21:18:10.589 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.582+0000 7f810ccd3700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f81081224c0 0x7f8108122ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:10.590 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.582+0000 7f8107fff700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f8108122f40 0x7f8108060110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:53554/0 (socket says 172.21.15.170:53554) 2020-03-19T21:18:10.590 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.582+0000 7f8107fff700 1 -- 172.21.15.170:0/1281801084 learned_addr learned my addr 172.21.15.170:0/1281801084 (peer_addr_for_me v1:172.21.15.170:0/0) 2020-03-19T21:18:10.590 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.583+0000 7f810ccd3700 1 -- 172.21.15.170:0/1281801084 >> v1:172.21.15.170:6789/0 conn(0x7f8108122f40 legacy=0x7f8108060110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:10.590 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.583+0000 7f810ccd3700 1 -- 172.21.15.170:0/1281801084 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8108090b60 con 0x7f81081224c0 2020-03-19T21:18:10.590 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.583+0000 7f810ccd3700 1 --2- 172.21.15.170:0/1281801084 >> v2:172.21.15.170:3300/0 conn(0x7f81081224c0 0x7f8108122ae0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f80f8002f30 tx=0x7f80f8007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:10.591 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.583+0000 7f810ccd3700 1 --2- 172.21.15.170:0/1281801084 >> v2:172.21.15.170:3300/0 conn(0x7f81081224c0 0x7f8108122ae0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f80f8002f30 tx=0x7f80f8007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:10.591 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.583+0000 7f810ccd3700 1 --2- 172.21.15.170:0/1281801084 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81081224c0 0x7f8108122ae0 secure :-1 s=READY pgs=67 cs=0 l=1 rx=0x7f80f8002f30 tx=0x7f80f8007a40).ready entity=mon.0 client_cookie=53210cbdf11444ca server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:10.591 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.583+0000 7f810ccd3700 1 --2- 172.21.15.170:0/1281801084 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81081224c0 0x7f8108122ae0 secure :-1 s=THROTTLE_DONE pgs=67 cs=0 l=1 rx=0x7f80f8002f30 tx=0x7f80f8007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:10.591 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.583+0000 7f81077fe700 1 -- 172.21.15.170:0/1281801084 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f80f8008e80 con 0x7f81081224c0 2020-03-19T21:18:10.591 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.583+0000 7f810ccd3700 1 --2- 172.21.15.170:0/1281801084 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81081224c0 0x7f8108122ae0 secure :-1 s=THROTTLE_DONE pgs=67 cs=0 l=1 rx=0x7f80f8002f30 tx=0x7f80f8007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:10.591 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.583+0000 7f81077fe700 1 -- 172.21.15.170:0/1281801084 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80f80113f0 con 0x7f81081224c0 2020-03-19T21:18:10.591 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.583+0000 7f810ccd3700 1 --2- 172.21.15.170:0/1281801084 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81081224c0 0x7f8108122ae0 secure :-1 s=THROTTLE_DONE pgs=67 cs=0 l=1 rx=0x7f80f8002f30 tx=0x7f80f8007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:10.592 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.583+0000 7f810ef37700 1 -- 172.21.15.170:0/1281801084 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81081224c0 msgr2=0x7f8108122ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:10.592 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.583+0000 7f810ef37700 1 --2- 172.21.15.170:0/1281801084 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81081224c0 0x7f8108122ae0 secure :-1 s=READY pgs=67 cs=0 l=1 rx=0x7f80f8002f30 tx=0x7f80f8007a40).stop 2020-03-19T21:18:10.592 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.583+0000 7f81077fe700 1 -- 172.21.15.170:0/1281801084 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f80f8011710 con 0x7f81081224c0 2020-03-19T21:18:10.592 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.583+0000 7f810ef37700 1 -- 172.21.15.170:0/1281801084 shutdown_connections 2020-03-19T21:18:10.592 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.583+0000 7f810ef37700 1 --2- 172.21.15.170:0/1281801084 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81081224c0 0x7f8108122ae0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:10.592 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.583+0000 7f810ef37700 1 -- 172.21.15.170:0/1281801084 >> 172.21.15.170:0/1281801084 conn(0x7f8108091390 msgr2=0x7f8108091770 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:10.592 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.583+0000 7f810ef37700 1 -- 172.21.15.170:0/1281801084 shutdown_connections 2020-03-19T21:18:10.593 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.583+0000 7f810ef37700 1 -- 172.21.15.170:0/1281801084 wait complete. 2020-03-19T21:18:10.593 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.584+0000 7f810ef37700 1 Processor -- start 2020-03-19T21:18:10.593 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.584+0000 7f810ef37700 1 -- start start 2020-03-19T21:18:10.593 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.585+0000 7f810ef37700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f81081224c0 0x7f810810b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:10.593 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.585+0000 7f810ef37700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8108123620 con 0x7f8108122f40 2020-03-19T21:18:10.593 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.585+0000 7f810ef37700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f810810d9f0 con 0x7f81081224c0 2020-03-19T21:18:10.593 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.585+0000 7f810ccd3700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f81081224c0 0x7f810810b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:10.594 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.585+0000 7f8107fff700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f8108122f40 0x7f810812fe90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:53560/0 (socket says 172.21.15.170:53560) 2020-03-19T21:18:10.594 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.585+0000 7f8107fff700 1 -- 172.21.15.170:0/1190891803 learned_addr learned my addr 172.21.15.170:0/1190891803 (peer_addr_for_me v1:172.21.15.170:0/0) 2020-03-19T21:18:10.594 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.585+0000 7f810ccd3700 1 --2- 172.21.15.170:0/1190891803 >> v2:172.21.15.170:3300/0 conn(0x7f81081224c0 0x7f810810b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:52794/0 (socket says 172.21.15.170:52794) 2020-03-19T21:18:10.594 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.585+0000 7f810ccd3700 1 -- 172.21.15.170:0/1190891803 >> v1:172.21.15.170:6789/0 conn(0x7f8108122f40 legacy=0x7f810812fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:10.594 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.585+0000 7f8105ffb700 1 -- 172.21.15.170:0/1190891803 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1128878746 0 0) 0x7f8108123620 con 0x7f8108122f40 2020-03-19T21:18:10.594 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.585+0000 7f810ccd3700 1 -- 172.21.15.170:0/1190891803 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8108123bd0 con 0x7f81081224c0 2020-03-19T21:18:10.594 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.585+0000 7f810ccd3700 1 --2- 172.21.15.170:0/1190891803 >> v2:172.21.15.170:3300/0 conn(0x7f81081224c0 0x7f810810b400 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f80f8002f30 tx=0x7f80f8007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:10.595 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.585+0000 7f810ccd3700 1 --2- 172.21.15.170:0/1190891803 >> v2:172.21.15.170:3300/0 conn(0x7f81081224c0 0x7f810810b400 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f80f8002f30 tx=0x7f80f8007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:10.595 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.585+0000 7f810ccd3700 1 --2- 172.21.15.170:0/1190891803 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81081224c0 0x7f810810b400 secure :-1 s=READY pgs=69 cs=0 l=1 rx=0x7f80f8002f30 tx=0x7f80f8007a40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:10.595 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.585+0000 7f810ccd3700 1 --2- 172.21.15.170:0/1190891803 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81081224c0 0x7f810810b400 secure :-1 s=THROTTLE_DONE pgs=69 cs=0 l=1 rx=0x7f80f8002f30 tx=0x7f80f8007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:10.595 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.585+0000 7f8105ffb700 1 -- 172.21.15.170:0/1190891803 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f80f800f160 con 0x7f81081224c0 2020-03-19T21:18:10.595 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.585+0000 7f810ccd3700 1 --2- 172.21.15.170:0/1190891803 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81081224c0 0x7f810810b400 secure :-1 s=THROTTLE_DONE pgs=69 cs=0 l=1 rx=0x7f80f8002f30 tx=0x7f80f8007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:10.595 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.585+0000 7f8105ffb700 1 -- 172.21.15.170:0/1190891803 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80f800f2c0 con 0x7f81081224c0 2020-03-19T21:18:10.595 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.585+0000 7f810ccd3700 1 --2- 172.21.15.170:0/1190891803 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81081224c0 0x7f810810b400 secure :-1 s=THROTTLE_DONE pgs=69 cs=0 l=1 rx=0x7f80f8002f30 tx=0x7f80f8007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:10.596 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.585+0000 7f810ef37700 1 -- 172.21.15.170:0/1190891803 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81081238c0 con 0x7f81081224c0 2020-03-19T21:18:10.596 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.585+0000 7f8105ffb700 1 -- 172.21.15.170:0/1190891803 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f80f80057f0 con 0x7f81081224c0 2020-03-19T21:18:10.596 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.585+0000 7f810ef37700 1 -- 172.21.15.170:0/1190891803 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f810811fce0 con 0x7f81081224c0 2020-03-19T21:18:10.596 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.586+0000 7f810ccd3700 1 --2- 172.21.15.170:0/1190891803 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81081224c0 0x7f810810b400 secure :-1 s=THROTTLE_DONE pgs=69 cs=0 l=1 rx=0x7f80f8002f30 tx=0x7f80f8007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:10.596 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.586+0000 7f8105ffb700 1 -- 172.21.15.170:0/1190891803 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 4) v1 ==== 29603+0+0 (secure 0 0 0) 0x7f80f8007180 con 0x7f81081224c0 2020-03-19T21:18:10.596 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.586+0000 7f810ccd3700 1 --2- 172.21.15.170:0/1190891803 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81081224c0 0x7f810810b400 secure :-1 s=THROTTLE_DONE pgs=69 cs=0 l=1 rx=0x7f80f8002f30 tx=0x7f80f8007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:10.596 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.586+0000 7f8105ffb700 1 --2- 172.21.15.170:0/1190891803 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f80e8028230 0x7f80e802a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:10.597 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.586+0000 7f8105ffb700 1 -- 172.21.15.170:0/1190891803 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(11..11 src has 1..11) v4 ==== 2583+0+0 (secure 0 0 0) 0x7f80f80364f0 con 0x7f81081224c0 2020-03-19T21:18:10.597 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.586+0000 7f8107fff700 1 --2- 172.21.15.170:0/1190891803 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f80e8028230 0x7f80e802a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:10.597 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.586+0000 7f810ef37700 1 -- 172.21.15.170:0/1190891803 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8108130760 con 0x7f81081224c0 2020-03-19T21:18:10.597 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.586+0000 7f8107fff700 1 --2- 172.21.15.170:0/1190891803 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f80e8028230 0x7f80e802a6f0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f80fc005500 tx=0x7f80fc001710).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:10.597 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.587+0000 7f8107fff700 1 --2- 172.21.15.170:0/1190891803 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f80e8028230 0x7f80e802a6f0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f80fc005500 tx=0x7f80fc001710).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:10.597 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.587+0000 7f8107fff700 1 --2- 172.21.15.170:0/1190891803 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f80e8028230 0x7f80e802a6f0 secure :-1 s=READY pgs=17 cs=0 l=1 rx=0x7f80fc005500 tx=0x7f80fc001710).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:10.598 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.590+0000 7f810ccd3700 1 --2- 172.21.15.170:0/1190891803 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81081224c0 0x7f810810b400 secure :-1 s=THROTTLE_DONE pgs=69 cs=0 l=1 rx=0x7f80f8002f30 tx=0x7f80f8007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:10.598 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.590+0000 7f8105ffb700 1 -- 172.21.15.170:0/1190891803 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+126772 (secure 0 0 0) 0x7f8108130760 con 0x7f81081224c0 2020-03-19T21:18:10.731 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.729+0000 7f810ef37700 1 -- 172.21.15.170:0/1190891803 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f8108090b60 con 0x7f81081224c0 2020-03-19T21:18:10.731 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.729+0000 7f810ccd3700 1 --2- 172.21.15.170:0/1190891803 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81081224c0 0x7f810810b400 secure :-1 s=THROTTLE_DONE pgs=69 cs=0 l=1 rx=0x7f80f8002f30 tx=0x7f80f8007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:10.732 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.729+0000 7f8105ffb700 1 -- 172.21.15.170:0/1190891803 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v11) v1 ==== 92+0+10 (secure 0 0 0) 0x7f8108090b60 con 0x7f81081224c0 2020-03-19T21:18:10.732 INFO:teuthology.orchestra.run.smithi170.stdout:pg_num: 8 2020-03-19T21:18:10.733 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.730+0000 7f810ef37700 1 -- 172.21.15.170:0/1190891803 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f80e8028230 msgr2=0x7f80e802a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:10.733 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.730+0000 7f810ef37700 1 --2- 172.21.15.170:0/1190891803 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f80e8028230 0x7f80e802a6f0 secure :-1 s=READY pgs=17 cs=0 l=1 rx=0x7f80fc005500 tx=0x7f80fc001710).stop 2020-03-19T21:18:10.733 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.730+0000 7f810ef37700 1 -- 172.21.15.170:0/1190891803 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81081224c0 msgr2=0x7f810810b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:10.733 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.730+0000 7f810ef37700 1 --2- 172.21.15.170:0/1190891803 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81081224c0 0x7f810810b400 secure :-1 s=READY pgs=69 cs=0 l=1 rx=0x7f80f8002f30 tx=0x7f80f8007a40).stop 2020-03-19T21:18:10.733 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.730+0000 7f810ef37700 1 -- 172.21.15.170:0/1190891803 shutdown_connections 2020-03-19T21:18:10.734 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.730+0000 7f810ef37700 1 --2- 172.21.15.170:0/1190891803 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f80e8028230 0x7f80e802a6f0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:10.734 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.730+0000 7f810ef37700 1 --2- 172.21.15.170:0/1190891803 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81081224c0 0x7f810810b400 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:10.734 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.730+0000 7f810ef37700 1 -- 172.21.15.170:0/1190891803 >> 172.21.15.170:0/1190891803 conn(0x7f8108091390 msgr2=0x7f8108093e00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:10.734 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.730+0000 7f810ef37700 1 -- 172.21.15.170:0/1190891803 shutdown_connections 2020-03-19T21:18:10.734 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.730+0000 7f810ef37700 1 -- 172.21.15.170:0/1190891803 wait complete. 2020-03-19T21:18:10.743 INFO:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early --format=json-pretty osd lspools 2020-03-19T21:18:10.848 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.846+0000 7f27bd047700 1 Processor -- start 2020-03-19T21:18:10.848 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.846+0000 7f27bd047700 1 -- start start 2020-03-19T21:18:10.854 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.846+0000 7f27bd047700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f27b80907c0 0x7f27b808daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T21:18:10.854 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.846+0000 7f27bd047700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27b80605e0 con 0x7f27b808d000 2020-03-19T21:18:10.855 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.846+0000 7f27bd047700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27b808ef70 con 0x7f27b80907c0 2020-03-19T21:18:10.855 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.846+0000 7f27ae59c700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f27b80907c0 0x7f27b808daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:10.856 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.846+0000 7f27b6d9d700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f27b808d000 0x7f27b808d3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:53564/0 (socket says 172.21.15.170:53564) 2020-03-19T21:18:10.856 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.846+0000 7f27b6d9d700 1 -- 172.21.15.170:0/587029571 learned_addr learned my addr 172.21.15.170:0/587029571 (peer_addr_for_me v1:172.21.15.170:0/0) 2020-03-19T21:18:10.856 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.847+0000 7f27ae59c700 1 -- 172.21.15.170:0/587029571 >> v1:172.21.15.170:6789/0 conn(0x7f27b808d000 legacy=0x7f27b808d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:10.856 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.847+0000 7f27b659c700 1 -- 172.21.15.170:0/587029571 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 252388437 0 0) 0x7f27b80605e0 con 0x7f27b808d000 2020-03-19T21:18:10.856 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.847+0000 7f27ae59c700 1 -- 172.21.15.170:0/587029571 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27b8060810 con 0x7f27b80907c0 2020-03-19T21:18:10.857 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.847+0000 7f27ae59c700 1 --2- 172.21.15.170:0/587029571 >> v2:172.21.15.170:3300/0 conn(0x7f27b80907c0 0x7f27b808daf0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f27b8090c70 tx=0x7f27a00088e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:10.857 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.847+0000 7f27ae59c700 1 --2- 172.21.15.170:0/587029571 >> v2:172.21.15.170:3300/0 conn(0x7f27b80907c0 0x7f27b808daf0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f27b8090c70 tx=0x7f27a00088e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:10.857 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.847+0000 7f27ae59c700 1 --2- 172.21.15.170:0/587029571 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f27b80907c0 0x7f27b808daf0 secure :-1 s=READY pgs=71 cs=0 l=1 rx=0x7f27b8090c70 tx=0x7f27a00088e0).ready entity=mon.0 client_cookie=3621fd430a6f467 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:10.857 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.847+0000 7f27ae59c700 1 --2- 172.21.15.170:0/587029571 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f27b80907c0 0x7f27b808daf0 secure :-1 s=THROTTLE_DONE pgs=71 cs=0 l=1 rx=0x7f27b8090c70 tx=0x7f27a00088e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:10.858 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.847+0000 7f27b659c700 1 -- 172.21.15.170:0/587029571 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f27a00113f0 con 0x7f27b80907c0 2020-03-19T21:18:10.858 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.847+0000 7f27ae59c700 1 --2- 172.21.15.170:0/587029571 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f27b80907c0 0x7f27b808daf0 secure :-1 s=THROTTLE_DONE pgs=71 cs=0 l=1 rx=0x7f27b8090c70 tx=0x7f27a00088e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:10.858 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.847+0000 7f27b659c700 1 -- 172.21.15.170:0/587029571 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27a0011550 con 0x7f27b80907c0 2020-03-19T21:18:10.858 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.847+0000 7f27ae59c700 1 --2- 172.21.15.170:0/587029571 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f27b80907c0 0x7f27b808daf0 secure :-1 s=THROTTLE_DONE pgs=71 cs=0 l=1 rx=0x7f27b8090c70 tx=0x7f27a00088e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:10.858 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.847+0000 7f27bd047700 1 -- 172.21.15.170:0/587029571 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f27b80907c0 msgr2=0x7f27b808daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:10.858 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.847+0000 7f27b659c700 1 -- 172.21.15.170:0/587029571 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f27a0011830 con 0x7f27b80907c0 2020-03-19T21:18:10.859 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.847+0000 7f27bd047700 1 --2- 172.21.15.170:0/587029571 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f27b80907c0 0x7f27b808daf0 secure :-1 s=READY pgs=71 cs=0 l=1 rx=0x7f27b8090c70 tx=0x7f27a00088e0).stop 2020-03-19T21:18:10.859 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.847+0000 7f27bd047700 1 -- 172.21.15.170:0/587029571 shutdown_connections 2020-03-19T21:18:10.859 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.847+0000 7f27bd047700 1 --2- 172.21.15.170:0/587029571 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f27b80907c0 0x7f27b808daf0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:10.859 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.848+0000 7f27bd047700 1 -- 172.21.15.170:0/587029571 >> 172.21.15.170:0/587029571 conn(0x7f27b8091540 msgr2=0x7f27b8091950 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:10.859 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.848+0000 7f27bd047700 1 -- 172.21.15.170:0/587029571 shutdown_connections 2020-03-19T21:18:10.859 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.848+0000 7f27bd047700 1 -- 172.21.15.170:0/587029571 wait complete. 2020-03-19T21:18:10.860 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.848+0000 7f27bd047700 1 Processor -- start 2020-03-19T21:18:10.860 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.848+0000 7f27bd047700 1 -- start start 2020-03-19T21:18:10.860 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.849+0000 7f27bd047700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f27b808d000 0x7f27b8122260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:10.860 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.849+0000 7f27bd047700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27b8121050 con 0x7f27b80907c0 2020-03-19T21:18:10.860 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.849+0000 7f27bd047700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27b811edf0 con 0x7f27b808d000 2020-03-19T21:18:10.860 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.849+0000 7f27b6d9d700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f27b808d000 0x7f27b8122260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:10.861 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.849+0000 7f27b6d9d700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f27b808d000 0x7f27b8122260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:52804/0 (socket says 172.21.15.170:52804) 2020-03-19T21:18:10.861 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.849+0000 7f27b6d9d700 1 -- 172.21.15.170:0/3483769297 learned_addr learned my addr 172.21.15.170:0/3483769297 (peer_addr_for_me v2:172.21.15.170:0/0) 2020-03-19T21:18:10.861 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.849+0000 7f27b6d9d700 1 -- 172.21.15.170:0/3483769297 >> v1:172.21.15.170:6789/0 conn(0x7f27b80907c0 legacy=0x7f27b81226c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:10.861 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.849+0000 7f27b4d99700 1 -- 172.21.15.170:0/3483769297 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1465047820 0 0) 0x7f27b8121050 con 0x7f27b80907c0 2020-03-19T21:18:10.861 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.849+0000 7f27b6d9d700 1 -- 172.21.15.170:0/3483769297 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27b8120d40 con 0x7f27b808d000 2020-03-19T21:18:10.861 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.849+0000 7f27b6d9d700 1 --2- 172.21.15.170:0/3483769297 >> v2:172.21.15.170:3300/0 conn(0x7f27b808d000 0x7f27b8122260 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f27b8120090 tx=0x7f27a8010990).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:10.861 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.849+0000 7f27b6d9d700 1 --2- 172.21.15.170:0/3483769297 >> v2:172.21.15.170:3300/0 conn(0x7f27b808d000 0x7f27b8122260 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f27b8120090 tx=0x7f27a8010990).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:10.862 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.849+0000 7f27b6d9d700 1 --2- 172.21.15.170:0/3483769297 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f27b808d000 0x7f27b8122260 secure :-1 s=READY pgs=73 cs=0 l=1 rx=0x7f27b8120090 tx=0x7f27a8010990).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:10.862 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.849+0000 7f27b6d9d700 1 --2- 172.21.15.170:0/3483769297 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f27b808d000 0x7f27b8122260 secure :-1 s=THROTTLE_DONE pgs=73 cs=0 l=1 rx=0x7f27b8120090 tx=0x7f27a8010990).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:10.862 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.849+0000 7f27b4d99700 1 -- 172.21.15.170:0/3483769297 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f27a8005b70 con 0x7f27b808d000 2020-03-19T21:18:10.862 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.849+0000 7f27b6d9d700 1 --2- 172.21.15.170:0/3483769297 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f27b808d000 0x7f27b8122260 secure :-1 s=THROTTLE_DONE pgs=73 cs=0 l=1 rx=0x7f27b8120090 tx=0x7f27a8010990).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:10.862 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.849+0000 7f27b6d9d700 1 --2- 172.21.15.170:0/3483769297 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f27b808d000 0x7f27b8122260 secure :-1 s=THROTTLE_DONE pgs=73 cs=0 l=1 rx=0x7f27b8120090 tx=0x7f27a8010990).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:10.862 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.849+0000 7f27b4d99700 1 -- 172.21.15.170:0/3483769297 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27a8005cd0 con 0x7f27b808d000 2020-03-19T21:18:10.862 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.849+0000 7f27bd047700 1 -- 172.21.15.170:0/3483769297 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27b808cb10 con 0x7f27b808d000 2020-03-19T21:18:10.863 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.849+0000 7f27b4d99700 1 -- 172.21.15.170:0/3483769297 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f27a80060c0 con 0x7f27b808d000 2020-03-19T21:18:10.863 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.849+0000 7f27bd047700 1 -- 172.21.15.170:0/3483769297 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27b811f240 con 0x7f27b808d000 2020-03-19T21:18:10.863 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.850+0000 7f27b6d9d700 1 --2- 172.21.15.170:0/3483769297 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f27b808d000 0x7f27b8122260 secure :-1 s=THROTTLE_DONE pgs=73 cs=0 l=1 rx=0x7f27b8120090 tx=0x7f27a8010990).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:10.863 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.850+0000 7f27b4d99700 1 -- 172.21.15.170:0/3483769297 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 4) v1 ==== 29603+0+0 (secure 0 0 0) 0x7f27a8006330 con 0x7f27b808d000 2020-03-19T21:18:10.863 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.850+0000 7f27b6d9d700 1 --2- 172.21.15.170:0/3483769297 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f27b808d000 0x7f27b8122260 secure :-1 s=THROTTLE_DONE pgs=73 cs=0 l=1 rx=0x7f27b8120090 tx=0x7f27a8010990).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:10.863 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.850+0000 7f27b4d99700 1 --2- 172.21.15.170:0/3483769297 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f279c028250 0x7f279c02a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:10.863 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.850+0000 7f27b4d99700 1 -- 172.21.15.170:0/3483769297 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(11..11 src has 1..11) v4 ==== 2583+0+0 (secure 0 0 0) 0x7f27a8033ff0 con 0x7f27b808d000 2020-03-19T21:18:10.864 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.850+0000 7f27ae59c700 1 --2- 172.21.15.170:0/3483769297 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f279c028250 0x7f279c02a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:10.864 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.850+0000 7f27bd047700 1 -- 172.21.15.170:0/3483769297 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27b81207e0 con 0x7f27b808d000 2020-03-19T21:18:10.864 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.851+0000 7f27ae59c700 1 --2- 172.21.15.170:0/3483769297 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f279c028250 0x7f279c02a710 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f27a00101c0 tx=0x7f27b8067850).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:10.864 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.851+0000 7f27ae59c700 1 --2- 172.21.15.170:0/3483769297 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f279c028250 0x7f279c02a710 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f27a00101c0 tx=0x7f27b8067850).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:10.864 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.851+0000 7f27ae59c700 1 --2- 172.21.15.170:0/3483769297 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f279c028250 0x7f279c02a710 secure :-1 s=READY pgs=18 cs=0 l=1 rx=0x7f27a00101c0 tx=0x7f27b8067850).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:10.865 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.854+0000 7f27b6d9d700 1 --2- 172.21.15.170:0/3483769297 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f27b808d000 0x7f27b8122260 secure :-1 s=THROTTLE_DONE pgs=73 cs=0 l=1 rx=0x7f27b8120090 tx=0x7f27a8010990).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:10.865 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.854+0000 7f27b4d99700 1 -- 172.21.15.170:0/3483769297 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+126772 (secure 0 0 0) 0x7f27a8007340 con 0x7f27b808d000 2020-03-19T21:18:10.998 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.995+0000 7f27bd047700 1 -- 172.21.15.170:0/3483769297 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f27b808c320 con 0x7f27b808d000 2020-03-19T21:18:10.998 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.996+0000 7f27b6d9d700 1 --2- 172.21.15.170:0/3483769297 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f27b808d000 0x7f27b8122260 secure :-1 s=THROTTLE_DONE pgs=73 cs=0 l=1 rx=0x7f27b8120090 tx=0x7f27a8010990).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:10.999 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.996+0000 7f27b4d99700 1 -- 172.21.15.170:0/3483769297 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v11) v1 ==== 84+0+64 (secure 0 0 0) 0x7f27a8007160 con 0x7f27b808d000 2020-03-19T21:18:10.999 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:18:10.999 INFO:teuthology.orchestra.run.smithi170.stdout:[ 2020-03-19T21:18:10.999 INFO:teuthology.orchestra.run.smithi170.stdout: { 2020-03-19T21:18:10.999 INFO:teuthology.orchestra.run.smithi170.stdout: "poolnum": 1, 2020-03-19T21:18:11.000 INFO:teuthology.orchestra.run.smithi170.stdout: "poolname": "rbd" 2020-03-19T21:18:11.000 INFO:teuthology.orchestra.run.smithi170.stdout: } 2020-03-19T21:18:11.000 INFO:teuthology.orchestra.run.smithi170.stdout:] 2020-03-19T21:18:11.001 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.997+0000 7f27bd047700 1 -- 172.21.15.170:0/3483769297 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f279c028250 msgr2=0x7f279c02a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:11.002 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.997+0000 7f27bd047700 1 --2- 172.21.15.170:0/3483769297 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f279c028250 0x7f279c02a710 secure :-1 s=READY pgs=18 cs=0 l=1 rx=0x7f27a00101c0 tx=0x7f27b8067850).stop 2020-03-19T21:18:11.002 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.997+0000 7f27bd047700 1 -- 172.21.15.170:0/3483769297 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f27b808d000 msgr2=0x7f27b8122260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:11.002 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.997+0000 7f27bd047700 1 --2- 172.21.15.170:0/3483769297 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f27b808d000 0x7f27b8122260 secure :-1 s=READY pgs=73 cs=0 l=1 rx=0x7f27b8120090 tx=0x7f27a8010990).stop 2020-03-19T21:18:11.002 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.997+0000 7f27bd047700 1 -- 172.21.15.170:0/3483769297 shutdown_connections 2020-03-19T21:18:11.002 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.997+0000 7f27bd047700 1 --2- 172.21.15.170:0/3483769297 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f279c028250 0x7f279c02a710 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:11.002 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.997+0000 7f27bd047700 1 --2- 172.21.15.170:0/3483769297 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f27b808d000 0x7f27b8122260 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:11.002 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.997+0000 7f27bd047700 1 -- 172.21.15.170:0/3483769297 >> 172.21.15.170:0/3483769297 conn(0x7f27b8091540 msgr2=0x7f27b80629b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:11.003 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.997+0000 7f27bd047700 1 -- 172.21.15.170:0/3483769297 shutdown_connections 2020-03-19T21:18:11.003 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:10.997+0000 7f27bd047700 1 -- 172.21.15.170:0/3483769297 wait complete. 2020-03-19T21:18:11.009 INFO:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2020-03-19T21:18:11.009 INFO:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool create cephfs_metadata 8 2020-03-19T21:18:11.117 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:11.115+0000 7efdf6873700 1 Processor -- start 2020-03-19T21:18:11.117 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:11.115+0000 7efdf6873700 1 -- start start 2020-03-19T21:18:11.124 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:11.116+0000 7efdf6873700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7efdf0120c30 0x7efdf0128960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T21:18:11.124 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:11.116+0000 7efdf6873700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efdf0091020 con 0x7efdf0091e80 2020-03-19T21:18:11.124 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:11.116+0000 7efdf6873700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdf0120020 con 0x7efdf0120c30 2020-03-19T21:18:11.124 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:11.116+0000 7efdef7fe700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7efdf0120c30 0x7efdf0128960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:11.124 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:11.116+0000 7efdeffff700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7efdf0091e80 0x7efdf0120520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:53574/0 (socket says 172.21.15.170:53574) 2020-03-19T21:18:11.124 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:11.116+0000 7efdeffff700 1 -- 172.21.15.170:0/532575410 learned_addr learned my addr 172.21.15.170:0/532575410 (peer_addr_for_me v1:172.21.15.170:0/0) 2020-03-19T21:18:11.125 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:11.116+0000 7efdef7fe700 1 -- 172.21.15.170:0/532575410 >> v1:172.21.15.170:6789/0 conn(0x7efdf0091e80 legacy=0x7efdf0120520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:11.125 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:11.116+0000 7efdef7fe700 1 -- 172.21.15.170:0/532575410 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdf0120210 con 0x7efdf0120c30 2020-03-19T21:18:11.125 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:11.116+0000 7efdeeffd700 1 -- 172.21.15.170:0/532575410 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3749680196 0 0) 0x7efdf0091020 con 0x7efdf0091e80 2020-03-19T21:18:11.125 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:11.116+0000 7efdef7fe700 1 --2- 172.21.15.170:0/532575410 >> v2:172.21.15.170:3300/0 conn(0x7efdf0120c30 0x7efdf0128960 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7efde4002f30 tx=0x7efde4007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:11.125 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:11.116+0000 7efdef7fe700 1 --2- 172.21.15.170:0/532575410 >> v2:172.21.15.170:3300/0 conn(0x7efdf0120c30 0x7efdf0128960 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7efde4002f30 tx=0x7efde4007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:11.125 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:11.116+0000 7efdef7fe700 1 --2- 172.21.15.170:0/532575410 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efdf0120c30 0x7efdf0128960 secure :-1 s=READY pgs=75 cs=0 l=1 rx=0x7efde4002f30 tx=0x7efde4007a40).ready entity=mon.0 client_cookie=c431a6561e7e012 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:11.125 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:11.116+0000 7efdef7fe700 1 --2- 172.21.15.170:0/532575410 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efdf0120c30 0x7efdf0128960 secure :-1 s=THROTTLE_DONE pgs=75 cs=0 l=1 rx=0x7efde4002f30 tx=0x7efde4007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:11.126 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:11.116+0000 7efdeeffd700 1 -- 172.21.15.170:0/532575410 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7efde4008e80 con 0x7efdf0120c30 2020-03-19T21:18:11.126 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:11.116+0000 7efdef7fe700 1 --2- 172.21.15.170:0/532575410 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efdf0120c30 0x7efdf0128960 secure :-1 s=THROTTLE_DONE pgs=75 cs=0 l=1 rx=0x7efde4002f30 tx=0x7efde4007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:11.126 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:11.116+0000 7efdef7fe700 1 --2- 172.21.15.170:0/532575410 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efdf0120c30 0x7efdf0128960 secure :-1 s=THROTTLE_DONE pgs=75 cs=0 l=1 rx=0x7efde4002f30 tx=0x7efde4007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:11.126 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:11.116+0000 7efdeeffd700 1 -- 172.21.15.170:0/532575410 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efde40113f0 con 0x7efdf0120c30 2020-03-19T21:18:11.126 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:11.116+0000 7efdeeffd700 1 -- 172.21.15.170:0/532575410 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7efde40116d0 con 0x7efdf0120c30 2020-03-19T21:18:11.126 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:11.116+0000 7efdf6873700 1 -- 172.21.15.170:0/532575410 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efdf0120c30 msgr2=0x7efdf0128960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:11.126 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:11.116+0000 7efdf6873700 1 --2- 172.21.15.170:0/532575410 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efdf0120c30 0x7efdf0128960 secure :-1 s=READY pgs=75 cs=0 l=1 rx=0x7efde4002f30 tx=0x7efde4007a40).stop 2020-03-19T21:18:11.127 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:11.117+0000 7efdf6873700 1 -- 172.21.15.170:0/532575410 shutdown_connections 2020-03-19T21:18:11.127 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:11.117+0000 7efdf6873700 1 --2- 172.21.15.170:0/532575410 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efdf0120c30 0x7efdf0128960 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:11.127 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:11.117+0000 7efdf6873700 1 -- 172.21.15.170:0/532575410 >> 172.21.15.170:0/532575410 conn(0x7efdf0091390 msgr2=0x7efdf0091770 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:11.127 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:11.117+0000 7efdf6873700 1 -- 172.21.15.170:0/532575410 shutdown_connections 2020-03-19T21:18:11.127 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:11.117+0000 7efdf6873700 1 -- 172.21.15.170:0/532575410 wait complete. 2020-03-19T21:18:11.127 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:11.117+0000 7efdf6873700 1 Processor -- start 2020-03-19T21:18:11.127 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:11.117+0000 7efdf6873700 1 -- start start 2020-03-19T21:18:11.128 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:11.118+0000 7efdf6873700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7efdf0120c30 0x7efdf011efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:11.128 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:11.118+0000 7efdf6873700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efdf010cb40 con 0x7efdf0091e80 2020-03-19T21:18:11.128 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:11.118+0000 7efdf6873700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdf010ef00 con 0x7efdf0120c30 2020-03-19T21:18:11.128 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:11.118+0000 7efdef7fe700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7efdf0120c30 0x7efdf011efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:11.128 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:11.118+0000 7efdeffff700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7efdf0091e80 0x7efdf011e890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:53578/0 (socket says 172.21.15.170:53578) 2020-03-19T21:18:11.128 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:11.118+0000 7efdeffff700 1 -- 172.21.15.170:0/3768815083 learned_addr learned my addr 172.21.15.170:0/3768815083 (peer_addr_for_me v1:172.21.15.170:0/0) 2020-03-19T21:18:11.128 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:11.118+0000 7efdef7fe700 1 -- 172.21.15.170:0/3768815083 >> v1:172.21.15.170:6789/0 conn(0x7efdf0091e80 legacy=0x7efdf011e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:11.129 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:11.118+0000 7efdef7fe700 1 -- 172.21.15.170:0/3768815083 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdf010f5a0 con 0x7efdf0120c30 2020-03-19T21:18:11.129 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:11.118+0000 7efded7fa700 1 -- 172.21.15.170:0/3768815083 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1627711416 0 0) 0x7efdf010cb40 con 0x7efdf0091e80 2020-03-19T21:18:11.129 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:11.118+0000 7efdef7fe700 1 --2- 172.21.15.170:0/3768815083 >> v2:172.21.15.170:3300/0 conn(0x7efdf0120c30 0x7efdf011efa0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7efdf010e9a0 tx=0x7efde4007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:11.129 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:11.118+0000 7efdef7fe700 1 --2- 172.21.15.170:0/3768815083 >> v2:172.21.15.170:3300/0 conn(0x7efdf0120c30 0x7efdf011efa0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7efdf010e9a0 tx=0x7efde4007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:11.129 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:11.118+0000 7efdef7fe700 1 --2- 172.21.15.170:0/3768815083 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efdf0120c30 0x7efdf011efa0 secure :-1 s=READY pgs=77 cs=0 l=1 rx=0x7efdf010e9a0 tx=0x7efde4007a40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:11.129 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:11.119+0000 7efdef7fe700 1 --2- 172.21.15.170:0/3768815083 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efdf0120c30 0x7efdf011efa0 secure :-1 s=THROTTLE_DONE pgs=77 cs=0 l=1 rx=0x7efdf010e9a0 tx=0x7efde4007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:11.129 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:11.119+0000 7efded7fa700 1 -- 172.21.15.170:0/3768815083 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7efde4011d80 con 0x7efdf0120c30 2020-03-19T21:18:11.130 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:11.119+0000 7efdef7fe700 1 --2- 172.21.15.170:0/3768815083 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efdf0120c30 0x7efdf011efa0 secure :-1 s=THROTTLE_DONE pgs=77 cs=0 l=1 rx=0x7efdf010e9a0 tx=0x7efde4007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:11.130 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:11.119+0000 7efdef7fe700 1 --2- 172.21.15.170:0/3768815083 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efdf0120c30 0x7efdf011efa0 secure :-1 s=THROTTLE_DONE pgs=77 cs=0 l=1 rx=0x7efdf010e9a0 tx=0x7efde4007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:11.130 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:11.119+0000 7efded7fa700 1 -- 172.21.15.170:0/3768815083 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efde4007150 con 0x7efdf0120c30 2020-03-19T21:18:11.130 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:11.119+0000 7efdf6873700 1 -- 172.21.15.170:0/3768815083 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdf010d700 con 0x7efdf0120c30 2020-03-19T21:18:11.130 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:11.119+0000 7efded7fa700 1 -- 172.21.15.170:0/3768815083 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7efde40073f0 con 0x7efdf0120c30 2020-03-19T21:18:11.130 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:11.119+0000 7efdf6873700 1 -- 172.21.15.170:0/3768815083 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdf010e2a0 con 0x7efdf0120c30 2020-03-19T21:18:11.131 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:11.119+0000 7efdef7fe700 1 --2- 172.21.15.170:0/3768815083 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efdf0120c30 0x7efdf011efa0 secure :-1 s=THROTTLE_DONE pgs=77 cs=0 l=1 rx=0x7efdf010e9a0 tx=0x7efde4007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:11.131 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:11.119+0000 7efdef7fe700 1 --2- 172.21.15.170:0/3768815083 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efdf0120c30 0x7efdf011efa0 secure :-1 s=THROTTLE_DONE pgs=77 cs=0 l=1 rx=0x7efdf010e9a0 tx=0x7efde4007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:11.131 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:11.119+0000 7efded7fa700 1 -- 172.21.15.170:0/3768815083 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 4) v1 ==== 29603+0+0 (secure 0 0 0) 0x7efde40056c0 con 0x7efdf0120c30 2020-03-19T21:18:11.131 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:11.119+0000 7efded7fa700 1 --2- 172.21.15.170:0/3768815083 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7efdd0028250 0x7efdd002a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:11.131 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:11.120+0000 7efded7fa700 1 -- 172.21.15.170:0/3768815083 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(11..11 src has 1..11) v4 ==== 2583+0+0 (secure 0 0 0) 0x7efde4034870 con 0x7efdf0120c30 2020-03-19T21:18:11.131 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:11.120+0000 7efdeffff700 1 --2- 172.21.15.170:0/3768815083 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7efdd0028250 0x7efdd002a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:11.132 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:11.120+0000 7efdf6873700 1 -- 172.21.15.170:0/3768815083 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdf0056db0 con 0x7efdf0120c30 2020-03-19T21:18:11.132 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:11.120+0000 7efdeffff700 1 --2- 172.21.15.170:0/3768815083 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7efdd0028250 0x7efdd002a710 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7efde0001260 tx=0x7efde0001720).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:11.132 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:11.120+0000 7efdeffff700 1 --2- 172.21.15.170:0/3768815083 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7efdd0028250 0x7efdd002a710 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7efde0001260 tx=0x7efde0001720).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:11.132 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:11.120+0000 7efdeffff700 1 --2- 172.21.15.170:0/3768815083 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7efdd0028250 0x7efdd002a710 secure :-1 s=READY pgs=19 cs=0 l=1 rx=0x7efde0001260 tx=0x7efde0001720).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:11.133 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:11.123+0000 7efdef7fe700 1 --2- 172.21.15.170:0/3768815083 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efdf0120c30 0x7efdf011efa0 secure :-1 s=THROTTLE_DONE pgs=77 cs=0 l=1 rx=0x7efdf010e9a0 tx=0x7efde4007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:11.133 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:11.123+0000 7efded7fa700 1 -- 172.21.15.170:0/3768815083 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+126772 (secure 0 0 0) 0x7efde4006070 con 0x7efdf0120c30 2020-03-19T21:18:11.266 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:11.264+0000 7efdf6873700 1 -- 172.21.15.170:0/3768815083 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7efdf0124430 con 0x7efdf0120c30 2020-03-19T21:18:12.121 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:12.119+0000 7efdef7fe700 1 --2- 172.21.15.170:0/3768815083 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efdf0120c30 0x7efdf011efa0 secure :-1 s=READY pgs=77 cs=0 l=1 rx=0x7efdf010e9a0 tx=0x7efde4007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:12.187 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:12.185+0000 7efdef7fe700 1 --2- 172.21.15.170:0/3768815083 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efdf0120c30 0x7efdf011efa0 secure :-1 s=THROTTLE_DONE pgs=77 cs=0 l=1 rx=0x7efdf010e9a0 tx=0x7efde4007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:12.187 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:12.185+0000 7efded7fa700 1 -- 172.21.15.170:0/3768815083 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v13) v1 ==== 133+0+0 (secure 0 0 0) 0x7efde4006070 con 0x7efdf0120c30 2020-03-19T21:18:12.191 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:12.189+0000 7efdf6873700 1 -- 172.21.15.170:0/3768815083 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7efdd0028250 msgr2=0x7efdd002a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:12.191 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:12.189+0000 7efdf6873700 1 --2- 172.21.15.170:0/3768815083 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7efdd0028250 0x7efdd002a710 secure :-1 s=READY pgs=19 cs=0 l=1 rx=0x7efde0001260 tx=0x7efde0001720).stop 2020-03-19T21:18:12.192 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:12.189+0000 7efdf6873700 1 -- 172.21.15.170:0/3768815083 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efdf0120c30 msgr2=0x7efdf011efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:12.192 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:12.189+0000 7efdf6873700 1 --2- 172.21.15.170:0/3768815083 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efdf0120c30 0x7efdf011efa0 secure :-1 s=READY pgs=77 cs=0 l=1 rx=0x7efdf010e9a0 tx=0x7efde4007a40).stop 2020-03-19T21:18:12.192 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:12.190+0000 7efdf6873700 1 -- 172.21.15.170:0/3768815083 shutdown_connections 2020-03-19T21:18:12.193 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:12.190+0000 7efdf6873700 1 --2- 172.21.15.170:0/3768815083 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7efdd0028250 0x7efdd002a710 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:12.193 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:12.190+0000 7efdf6873700 1 --2- 172.21.15.170:0/3768815083 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efdf0120c30 0x7efdf011efa0 secure :-1 s=CLOSED pgs=77 cs=0 l=1 rx=0x7efdf010e9a0 tx=0x7efde4007a40).stop 2020-03-19T21:18:12.193 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:12.190+0000 7efdf6873700 1 -- 172.21.15.170:0/3768815083 >> 172.21.15.170:0/3768815083 conn(0x7efdf0091390 msgr2=0x7efdf008f290 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:12.193 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:12.190+0000 7efdf6873700 1 -- 172.21.15.170:0/3768815083 shutdown_connections 2020-03-19T21:18:12.193 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:12.190+0000 7efdf6873700 1 -- 172.21.15.170:0/3768815083 wait complete. 2020-03-19T21:18:12.194 INFO:teuthology.orchestra.run.smithi170.stderr:pool 'cephfs_metadata' created 2020-03-19T21:18:12.204 INFO:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool create cephfs_data 8 2020-03-19T21:18:12.429 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:12.426+0000 7f48f51e8700 1 Processor -- start 2020-03-19T21:18:12.429 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:12.426+0000 7f48f51e8700 1 -- start start 2020-03-19T21:18:12.429 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:12.426+0000 7f48f51e8700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f48f0066030 0x7f48f0127070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T21:18:12.429 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:12.426+0000 7f48f51e8700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48f0061300 con 0x7f48f0065540 2020-03-19T21:18:12.429 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:12.426+0000 7f48f51e8700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48f008e940 con 0x7f48f0066030 2020-03-19T21:18:12.430 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:12.426+0000 7f48effff700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f48f0065540 0x7f48f0065920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:53584/0 (socket says 172.21.15.170:53584) 2020-03-19T21:18:12.430 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:12.426+0000 7f48effff700 1 -- 172.21.15.170:0/790079532 learned_addr learned my addr 172.21.15.170:0/790079532 (peer_addr_for_me v1:172.21.15.170:0/0) 2020-03-19T21:18:12.431 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:12.426+0000 7f48e7fff700 1 --2- 172.21.15.170:0/790079532 >> v2:172.21.15.170:3300/0 conn(0x7f48f0066030 0x7f48f0127070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:12.431 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:12.427+0000 7f48ef7fe700 1 -- 172.21.15.170:0/790079532 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 132309291 0 0) 0x7f48f0061300 con 0x7f48f0065540 2020-03-19T21:18:12.431 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:12.427+0000 7f48ef7fe700 1 -- 172.21.15.170:0/790079532 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f48d0003230 con 0x7f48f0065540 2020-03-19T21:18:12.431 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:12.427+0000 7f48e7fff700 1 -- 172.21.15.170:0/790079532 >> v1:172.21.15.170:6789/0 conn(0x7f48f0065540 legacy=0x7f48f0065920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:12.431 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:12.427+0000 7f48e7fff700 1 -- 172.21.15.170:0/790079532 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48f0060fc0 con 0x7f48f0066030 2020-03-19T21:18:12.441 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:12.427+0000 7f48e7fff700 1 --2- 172.21.15.170:0/790079532 >> v2:172.21.15.170:3300/0 conn(0x7f48f0066030 0x7f48f0127070 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f48d8002f30 tx=0x7f48d8007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:12.441 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:12.427+0000 7f48e7fff700 1 --2- 172.21.15.170:0/790079532 >> v2:172.21.15.170:3300/0 conn(0x7f48f0066030 0x7f48f0127070 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f48d8002f30 tx=0x7f48d8007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:12.442 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:12.427+0000 7f48e7fff700 1 --2- 172.21.15.170:0/790079532 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48f0066030 0x7f48f0127070 secure :-1 s=READY pgs=79 cs=0 l=1 rx=0x7f48d8002f30 tx=0x7f48d8007a40).ready entity=mon.0 client_cookie=d0697f313a983d6e server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:12.442 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:12.427+0000 7f48e7fff700 1 --2- 172.21.15.170:0/790079532 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48f0066030 0x7f48f0127070 secure :-1 s=THROTTLE_DONE pgs=79 cs=0 l=1 rx=0x7f48d8002f30 tx=0x7f48d8007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:12.442 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:12.427+0000 7f48ef7fe700 1 -- 172.21.15.170:0/790079532 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f48d8011340 con 0x7f48f0066030 2020-03-19T21:18:12.442 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:12.427+0000 7f48e7fff700 1 --2- 172.21.15.170:0/790079532 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48f0066030 0x7f48f0127070 secure :-1 s=THROTTLE_DONE pgs=79 cs=0 l=1 rx=0x7f48d8002f30 tx=0x7f48d8007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:12.442 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:12.427+0000 7f48e7fff700 1 --2- 172.21.15.170:0/790079532 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48f0066030 0x7f48f0127070 secure :-1 s=THROTTLE_DONE pgs=79 cs=0 l=1 rx=0x7f48d8002f30 tx=0x7f48d8007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:12.442 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:12.427+0000 7f48ef7fe700 1 -- 172.21.15.170:0/790079532 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48d80114a0 con 0x7f48f0066030 2020-03-19T21:18:12.443 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:12.427+0000 7f48ef7fe700 1 -- 172.21.15.170:0/790079532 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f48d80117c0 con 0x7f48f0066030 2020-03-19T21:18:12.443 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:12.427+0000 7f48f51e8700 1 -- 172.21.15.170:0/790079532 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48f0066030 msgr2=0x7f48f0127070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:12.443 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:12.427+0000 7f48f51e8700 1 --2- 172.21.15.170:0/790079532 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48f0066030 0x7f48f0127070 secure :-1 s=READY pgs=79 cs=0 l=1 rx=0x7f48d8002f30 tx=0x7f48d8007a40).stop 2020-03-19T21:18:12.443 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:12.428+0000 7f48f51e8700 1 -- 172.21.15.170:0/790079532 shutdown_connections 2020-03-19T21:18:12.443 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:12.428+0000 7f48f51e8700 1 --2- 172.21.15.170:0/790079532 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48f0066030 0x7f48f0127070 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:12.443 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:12.428+0000 7f48f51e8700 1 -- 172.21.15.170:0/790079532 >> 172.21.15.170:0/790079532 conn(0x7f48f00950e0 msgr2=0x7f48f0091180 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:12.443 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:12.428+0000 7f48f51e8700 1 -- 172.21.15.170:0/790079532 shutdown_connections 2020-03-19T21:18:12.444 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:12.428+0000 7f48f51e8700 1 -- 172.21.15.170:0/790079532 wait complete. 2020-03-19T21:18:12.444 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:12.428+0000 7f48f51e8700 1 Processor -- start 2020-03-19T21:18:12.444 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:12.428+0000 7f48f51e8700 1 -- start start 2020-03-19T21:18:12.444 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:12.429+0000 7f48f51e8700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f48f0065540 0x7f48f012e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:12.444 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:12.429+0000 7f48f51e8700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48f01228d0 con 0x7f48f0066030 2020-03-19T21:18:12.444 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:12.429+0000 7f48f51e8700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48f0123240 con 0x7f48f0065540 2020-03-19T21:18:12.445 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:12.429+0000 7f48effff700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f48f0065540 0x7f48f012e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:12.445 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:12.429+0000 7f48e7fff700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f48f0066030 0x7f48f012e950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:53590/0 (socket says 172.21.15.170:53590) 2020-03-19T21:18:12.445 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:12.429+0000 7f48e7fff700 1 -- 172.21.15.170:0/1772426537 learned_addr learned my addr 172.21.15.170:0/1772426537 (peer_addr_for_me v1:172.21.15.170:0/0) 2020-03-19T21:18:12.445 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:12.429+0000 7f48effff700 1 -- 172.21.15.170:0/1772426537 >> v1:172.21.15.170:6789/0 conn(0x7f48f0066030 legacy=0x7f48f012e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:12.445 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:12.429+0000 7f48edffb700 1 -- 172.21.15.170:0/1772426537 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3845250995 0 0) 0x7f48f0060fc0 con 0x7f48f0066030 2020-03-19T21:18:12.445 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:12.429+0000 7f48effff700 1 -- 172.21.15.170:0/1772426537 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48f0122f30 con 0x7f48f0065540 2020-03-19T21:18:12.445 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:12.429+0000 7f48effff700 1 --2- 172.21.15.170:0/1772426537 >> v2:172.21.15.170:3300/0 conn(0x7f48f0065540 0x7f48f012e4f0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f48e00010d0 tx=0x7f48e0004910).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:12.446 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:12.429+0000 7f48effff700 1 --2- 172.21.15.170:0/1772426537 >> v2:172.21.15.170:3300/0 conn(0x7f48f0065540 0x7f48f012e4f0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f48e00010d0 tx=0x7f48e0004910).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:12.446 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:12.429+0000 7f48effff700 1 --2- 172.21.15.170:0/1772426537 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48f0065540 0x7f48f012e4f0 secure :-1 s=READY pgs=81 cs=0 l=1 rx=0x7f48e00010d0 tx=0x7f48e0004910).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:12.446 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:12.429+0000 7f48effff700 1 --2- 172.21.15.170:0/1772426537 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48f0065540 0x7f48f012e4f0 secure :-1 s=THROTTLE_DONE pgs=81 cs=0 l=1 rx=0x7f48e00010d0 tx=0x7f48e0004910).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:12.446 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:12.429+0000 7f48edffb700 1 -- 172.21.15.170:0/1772426537 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f48e0013e40 con 0x7f48f0065540 2020-03-19T21:18:12.446 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:12.429+0000 7f48effff700 1 --2- 172.21.15.170:0/1772426537 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48f0065540 0x7f48f012e4f0 secure :-1 s=THROTTLE_DONE pgs=81 cs=0 l=1 rx=0x7f48e00010d0 tx=0x7f48e0004910).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:12.446 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:12.429+0000 7f48edffb700 1 -- 172.21.15.170:0/1772426537 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48e00066f0 con 0x7f48f0065540 2020-03-19T21:18:12.447 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:12.429+0000 7f48effff700 1 --2- 172.21.15.170:0/1772426537 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48f0065540 0x7f48f012e4f0 secure :-1 s=THROTTLE_DONE pgs=81 cs=0 l=1 rx=0x7f48e00010d0 tx=0x7f48e0004910).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:12.447 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:12.429+0000 7f48f51e8700 1 -- 172.21.15.170:0/1772426537 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48f010eda0 con 0x7f48f0065540 2020-03-19T21:18:12.448 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:12.429+0000 7f48edffb700 1 -- 172.21.15.170:0/1772426537 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f48e00069d0 con 0x7f48f0065540 2020-03-19T21:18:12.449 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:12.430+0000 7f48f51e8700 1 -- 172.21.15.170:0/1772426537 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48f010d5c0 con 0x7f48f0065540 2020-03-19T21:18:12.449 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:12.430+0000 7f48effff700 1 --2- 172.21.15.170:0/1772426537 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48f0065540 0x7f48f012e4f0 secure :-1 s=THROTTLE_DONE pgs=81 cs=0 l=1 rx=0x7f48e00010d0 tx=0x7f48e0004910).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:12.449 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:12.430+0000 7f48effff700 1 --2- 172.21.15.170:0/1772426537 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48f0065540 0x7f48f012e4f0 secure :-1 s=THROTTLE_DONE pgs=81 cs=0 l=1 rx=0x7f48e00010d0 tx=0x7f48e0004910).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:12.449 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:12.430+0000 7f48edffb700 1 -- 172.21.15.170:0/1772426537 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 4) v1 ==== 29603+0+0 (secure 0 0 0) 0x7f48e0006bd0 con 0x7f48f0065540 2020-03-19T21:18:12.449 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:12.430+0000 7f48edffb700 1 --2- 172.21.15.170:0/1772426537 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f48d4028230 0x7f48d402a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:12.449 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:12.430+0000 7f48edffb700 1 -- 172.21.15.170:0/1772426537 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 2937+0+0 (secure 0 0 0) 0x7f48e0035f60 con 0x7f48f0065540 2020-03-19T21:18:12.450 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:12.430+0000 7f48e7fff700 1 --2- 172.21.15.170:0/1772426537 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f48d4028230 0x7f48d402a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:12.450 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:12.431+0000 7f48f51e8700 1 -- 172.21.15.170:0/1772426537 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48f0056db0 con 0x7f48f0065540 2020-03-19T21:18:12.450 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:12.431+0000 7f48e7fff700 1 --2- 172.21.15.170:0/1772426537 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f48d4028230 0x7f48d402a6f0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f48d8010110 tx=0x7f48d8007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:12.450 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:12.431+0000 7f48e7fff700 1 --2- 172.21.15.170:0/1772426537 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f48d4028230 0x7f48d402a6f0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f48d8010110 tx=0x7f48d8007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:12.450 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:12.431+0000 7f48e7fff700 1 --2- 172.21.15.170:0/1772426537 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f48d4028230 0x7f48d402a6f0 secure :-1 s=READY pgs=20 cs=0 l=1 rx=0x7f48d8010110 tx=0x7f48d8007a40).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:12.450 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:12.434+0000 7f48effff700 1 --2- 172.21.15.170:0/1772426537 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48f0065540 0x7f48f012e4f0 secure :-1 s=THROTTLE_DONE pgs=81 cs=0 l=1 rx=0x7f48e00010d0 tx=0x7f48e0004910).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:12.450 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:12.434+0000 7f48edffb700 1 -- 172.21.15.170:0/1772426537 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+126772 (secure 0 0 0) 0x7f48e00073f0 con 0x7f48f0065540 2020-03-19T21:18:12.577 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:12.574+0000 7f48f51e8700 1 -- 172.21.15.170:0/1772426537 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7f48f005e820 con 0x7f48f0065540 2020-03-19T21:18:13.191 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.189+0000 7f48effff700 1 --2- 172.21.15.170:0/1772426537 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48f0065540 0x7f48f012e4f0 secure :-1 s=THROTTLE_DONE pgs=81 cs=0 l=1 rx=0x7f48e00010d0 tx=0x7f48e0004910).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:13.192 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.190+0000 7f48edffb700 1 -- 172.21.15.170:0/1772426537 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v14) v1 ==== 125+0+0 (secure 0 0 0) 0x7f48e00073f0 con 0x7f48f0065540 2020-03-19T21:18:13.197 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.195+0000 7f48f51e8700 1 -- 172.21.15.170:0/1772426537 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f48d4028230 msgr2=0x7f48d402a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:13.197 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.195+0000 7f48f51e8700 1 --2- 172.21.15.170:0/1772426537 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f48d4028230 0x7f48d402a6f0 secure :-1 s=READY pgs=20 cs=0 l=1 rx=0x7f48d8010110 tx=0x7f48d8007a40).stop 2020-03-19T21:18:13.198 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.195+0000 7f48f51e8700 1 -- 172.21.15.170:0/1772426537 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48f0065540 msgr2=0x7f48f012e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:13.198 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.195+0000 7f48f51e8700 1 --2- 172.21.15.170:0/1772426537 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48f0065540 0x7f48f012e4f0 secure :-1 s=READY pgs=81 cs=0 l=1 rx=0x7f48e00010d0 tx=0x7f48e0004910).stop 2020-03-19T21:18:13.198 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.196+0000 7f48f51e8700 1 -- 172.21.15.170:0/1772426537 shutdown_connections 2020-03-19T21:18:13.199 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.196+0000 7f48f51e8700 1 --2- 172.21.15.170:0/1772426537 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f48d4028230 0x7f48d402a6f0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:13.199 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.196+0000 7f48f51e8700 1 --2- 172.21.15.170:0/1772426537 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48f0065540 0x7f48f012e4f0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:13.199 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.196+0000 7f48f51e8700 1 -- 172.21.15.170:0/1772426537 >> 172.21.15.170:0/1772426537 conn(0x7f48f00950e0 msgr2=0x7f48f0093320 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:13.199 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.196+0000 7f48f51e8700 1 -- 172.21.15.170:0/1772426537 shutdown_connections 2020-03-19T21:18:13.199 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.196+0000 7f48f51e8700 1 -- 172.21.15.170:0/1772426537 wait complete. 2020-03-19T21:18:13.199 INFO:teuthology.orchestra.run.smithi170.stderr:pool 'cephfs_data' created 2020-03-19T21:18:13.210 INFO:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs new cephfs cephfs_metadata cephfs_data --force 2020-03-19T21:18:13.320 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.318+0000 7f096f711700 1 Processor -- start 2020-03-19T21:18:13.321 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.318+0000 7f096f711700 1 -- start start 2020-03-19T21:18:13.326 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.319+0000 7f096f711700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f0968125840 0x7f0968127c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T21:18:13.326 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.319+0000 7f096f711700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09681228a0 con 0x7f09681224c0 2020-03-19T21:18:13.326 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.319+0000 7f096f711700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0968092200 con 0x7f0968125840 2020-03-19T21:18:13.326 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.319+0000 7f096ccac700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f0968125840 0x7f0968127c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:13.326 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.319+0000 7f096d4ad700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f09681224c0 0x7f0968122ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:53594/0 (socket says 172.21.15.170:53594) 2020-03-19T21:18:13.326 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.319+0000 7f096d4ad700 1 -- 172.21.15.170:0/959299997 learned_addr learned my addr 172.21.15.170:0/959299997 (peer_addr_for_me v1:172.21.15.170:0/0) 2020-03-19T21:18:13.326 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.319+0000 7f096ccac700 1 -- 172.21.15.170:0/959299997 >> v1:172.21.15.170:6789/0 conn(0x7f09681224c0 legacy=0x7f0968122ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:13.327 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.319+0000 7f095bfff700 1 -- 172.21.15.170:0/959299997 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3570336974 0 0) 0x7f095c0010d0 con 0x7f09681224c0 2020-03-19T21:18:13.327 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.319+0000 7f096ccac700 1 -- 172.21.15.170:0/959299997 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0968090b60 con 0x7f0968125840 2020-03-19T21:18:13.327 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.319+0000 7f096ccac700 1 --2- 172.21.15.170:0/959299997 >> v2:172.21.15.170:3300/0 conn(0x7f0968125840 0x7f0968127c30 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f0964002f30 tx=0x7f0964007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:13.327 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.319+0000 7f096ccac700 1 --2- 172.21.15.170:0/959299997 >> v2:172.21.15.170:3300/0 conn(0x7f0968125840 0x7f0968127c30 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f0964002f30 tx=0x7f0964007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:13.327 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.319+0000 7f096ccac700 1 --2- 172.21.15.170:0/959299997 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0968125840 0x7f0968127c30 secure :-1 s=READY pgs=83 cs=0 l=1 rx=0x7f0964002f30 tx=0x7f0964007a40).ready entity=mon.0 client_cookie=37e9e964dc0de98 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:13.327 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.320+0000 7f096ccac700 1 --2- 172.21.15.170:0/959299997 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0968125840 0x7f0968127c30 secure :-1 s=THROTTLE_DONE pgs=83 cs=0 l=1 rx=0x7f0964002f30 tx=0x7f0964007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:13.327 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.320+0000 7f095bfff700 1 -- 172.21.15.170:0/959299997 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f0964008e80 con 0x7f0968125840 2020-03-19T21:18:13.328 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.320+0000 7f096ccac700 1 --2- 172.21.15.170:0/959299997 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0968125840 0x7f0968127c30 secure :-1 s=THROTTLE_DONE pgs=83 cs=0 l=1 rx=0x7f0964002f30 tx=0x7f0964007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:13.328 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.320+0000 7f095bfff700 1 -- 172.21.15.170:0/959299997 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09640113f0 con 0x7f0968125840 2020-03-19T21:18:13.328 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.320+0000 7f096ccac700 1 --2- 172.21.15.170:0/959299997 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0968125840 0x7f0968127c30 secure :-1 s=THROTTLE_DONE pgs=83 cs=0 l=1 rx=0x7f0964002f30 tx=0x7f0964007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:13.328 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.320+0000 7f095bfff700 1 -- 172.21.15.170:0/959299997 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f09640116d0 con 0x7f0968125840 2020-03-19T21:18:13.328 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.320+0000 7f096f711700 1 -- 172.21.15.170:0/959299997 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0968125840 msgr2=0x7f0968127c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:13.328 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.320+0000 7f096f711700 1 --2- 172.21.15.170:0/959299997 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0968125840 0x7f0968127c30 secure :-1 s=READY pgs=83 cs=0 l=1 rx=0x7f0964002f30 tx=0x7f0964007a40).stop 2020-03-19T21:18:13.328 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.320+0000 7f096f711700 1 -- 172.21.15.170:0/959299997 shutdown_connections 2020-03-19T21:18:13.328 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.320+0000 7f096f711700 1 --2- 172.21.15.170:0/959299997 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0968125840 0x7f0968127c30 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:13.329 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.320+0000 7f096f711700 1 -- 172.21.15.170:0/959299997 >> 172.21.15.170:0/959299997 conn(0x7f0968091390 msgr2=0x7f0968091770 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:13.329 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.320+0000 7f096f711700 1 -- 172.21.15.170:0/959299997 shutdown_connections 2020-03-19T21:18:13.329 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.320+0000 7f096f711700 1 -- 172.21.15.170:0/959299997 wait complete. 2020-03-19T21:18:13.329 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.321+0000 7f096f711700 1 Processor -- start 2020-03-19T21:18:13.329 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.321+0000 7f096f711700 1 -- start start 2020-03-19T21:18:13.329 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.321+0000 7f096f711700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f0968125840 0x7f09680664e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:13.330 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.321+0000 7f096f711700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0968065a30 con 0x7f09681224c0 2020-03-19T21:18:13.330 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.321+0000 7f096f711700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0968123620 con 0x7f0968125840 2020-03-19T21:18:13.330 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.321+0000 7f096d4ad700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f09681224c0 0x7f0968061c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:53598/0 (socket says 172.21.15.170:53598) 2020-03-19T21:18:13.330 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.321+0000 7f096d4ad700 1 -- 172.21.15.170:0/1629566839 learned_addr learned my addr 172.21.15.170:0/1629566839 (peer_addr_for_me v1:172.21.15.170:0/0) 2020-03-19T21:18:13.330 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.321+0000 7f096ccac700 1 --2- 172.21.15.170:0/1629566839 >> v2:172.21.15.170:3300/0 conn(0x7f0968125840 0x7f09680664e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:13.330 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.321+0000 7f095a7fc700 1 -- 172.21.15.170:0/1629566839 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3040271663 0 0) 0x7f0968065a30 con 0x7f09681224c0 2020-03-19T21:18:13.331 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.322+0000 7f095a7fc700 1 -- 172.21.15.170:0/1629566839 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0948002fe0 con 0x7f09681224c0 2020-03-19T21:18:13.331 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.322+0000 7f096ccac700 1 -- 172.21.15.170:0/1629566839 >> v1:172.21.15.170:6789/0 conn(0x7f09681224c0 legacy=0x7f0968061c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:13.331 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.322+0000 7f096ccac700 1 -- 172.21.15.170:0/1629566839 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0968066940 con 0x7f0968125840 2020-03-19T21:18:13.331 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.322+0000 7f096ccac700 1 --2- 172.21.15.170:0/1629566839 >> v2:172.21.15.170:3300/0 conn(0x7f0968125840 0x7f09680664e0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f0964002f30 tx=0x7f0964007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:13.331 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.322+0000 7f096ccac700 1 --2- 172.21.15.170:0/1629566839 >> v2:172.21.15.170:3300/0 conn(0x7f0968125840 0x7f09680664e0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f0964002f30 tx=0x7f0964007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:13.331 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.322+0000 7f096ccac700 1 --2- 172.21.15.170:0/1629566839 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0968125840 0x7f09680664e0 secure :-1 s=READY pgs=85 cs=0 l=1 rx=0x7f0964002f30 tx=0x7f0964007a40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:13.331 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.322+0000 7f096ccac700 1 --2- 172.21.15.170:0/1629566839 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0968125840 0x7f09680664e0 secure :-1 s=THROTTLE_DONE pgs=85 cs=0 l=1 rx=0x7f0964002f30 tx=0x7f0964007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:13.332 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.322+0000 7f095a7fc700 1 -- 172.21.15.170:0/1629566839 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f0964011d80 con 0x7f0968125840 2020-03-19T21:18:13.332 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.322+0000 7f096ccac700 1 --2- 172.21.15.170:0/1629566839 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0968125840 0x7f09680664e0 secure :-1 s=THROTTLE_DONE pgs=85 cs=0 l=1 rx=0x7f0964002f30 tx=0x7f0964007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:13.332 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.322+0000 7f095a7fc700 1 -- 172.21.15.170:0/1629566839 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0964007150 con 0x7f0968125840 2020-03-19T21:18:13.332 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.322+0000 7f096ccac700 1 --2- 172.21.15.170:0/1629566839 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0968125840 0x7f09680664e0 secure :-1 s=THROTTLE_DONE pgs=85 cs=0 l=1 rx=0x7f0964002f30 tx=0x7f0964007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:13.332 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.322+0000 7f095a7fc700 1 -- 172.21.15.170:0/1629566839 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f0964007470 con 0x7f0968125840 2020-03-19T21:18:13.332 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.322+0000 7f096f711700 1 -- 172.21.15.170:0/1629566839 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09680633b0 con 0x7f0968125840 2020-03-19T21:18:13.332 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.322+0000 7f096f711700 1 -- 172.21.15.170:0/1629566839 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0968064090 con 0x7f0968125840 2020-03-19T21:18:13.333 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.322+0000 7f096ccac700 1 --2- 172.21.15.170:0/1629566839 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0968125840 0x7f09680664e0 secure :-1 s=THROTTLE_DONE pgs=85 cs=0 l=1 rx=0x7f0964002f30 tx=0x7f0964007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:13.333 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.323+0000 7f095a7fc700 1 -- 172.21.15.170:0/1629566839 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 4) v1 ==== 29603+0+0 (secure 0 0 0) 0x7f09640056c0 con 0x7f0968125840 2020-03-19T21:18:13.333 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.323+0000 7f096ccac700 1 --2- 172.21.15.170:0/1629566839 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0968125840 0x7f09680664e0 secure :-1 s=THROTTLE_DONE pgs=85 cs=0 l=1 rx=0x7f0964002f30 tx=0x7f0964007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:13.333 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.323+0000 7f095a7fc700 1 --2- 172.21.15.170:0/1629566839 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f09480287b0 0x7f094802ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:13.333 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.323+0000 7f095a7fc700 1 -- 172.21.15.170:0/1629566839 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 3287+0+0 (secure 0 0 0) 0x7f0964034680 con 0x7f0968125840 2020-03-19T21:18:13.333 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.323+0000 7f096d4ad700 1 --2- 172.21.15.170:0/1629566839 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f09480287b0 0x7f094802ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:13.333 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.323+0000 7f096d4ad700 1 --2- 172.21.15.170:0/1629566839 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f09480287b0 0x7f094802ac70 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f095c000ff0 tx=0x7f095c0016c0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:13.333 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.323+0000 7f096d4ad700 1 --2- 172.21.15.170:0/1629566839 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f09480287b0 0x7f094802ac70 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f095c000ff0 tx=0x7f095c0016c0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:13.334 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.323+0000 7f096d4ad700 1 --2- 172.21.15.170:0/1629566839 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f09480287b0 0x7f094802ac70 secure :-1 s=READY pgs=21 cs=0 l=1 rx=0x7f095c000ff0 tx=0x7f095c0016c0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:13.334 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.323+0000 7f096f711700 1 -- 172.21.15.170:0/1629566839 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0968090b60 con 0x7f0968125840 2020-03-19T21:18:13.334 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.327+0000 7f096ccac700 1 --2- 172.21.15.170:0/1629566839 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0968125840 0x7f09680664e0 secure :-1 s=THROTTLE_DONE pgs=85 cs=0 l=1 rx=0x7f0964002f30 tx=0x7f0964007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:13.334 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.327+0000 7f095a7fc700 1 -- 172.21.15.170:0/1629566839 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+126772 (secure 0 0 0) 0x7f0968090b60 con 0x7f0968125840 2020-03-19T21:18:13.514 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.512+0000 7f096f711700 1 -- 172.21.15.170:0/1629566839 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7f0968090b60 con 0x7f0968125840 2020-03-19T21:18:13.515 INFO:tasks.ceph.mon.a.smithi170.stderr:2020-03-19T21:18:13.513+0000 7f6faba09700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2020-03-19T21:18:13.520 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.518+0000 7f096ccac700 1 --2- 172.21.15.170:0/1629566839 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0968125840 0x7f09680664e0 secure :-1 s=THROTTLE_DONE pgs=85 cs=0 l=1 rx=0x7f0964002f30 tx=0x7f0964007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:13.522 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.519+0000 7f095a7fc700 1 -- 172.21.15.170:0/1629566839 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 2 and data pool 3 v2) v1 ==== 187+0+0 (secure 0 0 0) 0x7f0968090b60 con 0x7f0968125840 2020-03-19T21:18:13.525 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.523+0000 7f096f711700 1 -- 172.21.15.170:0/1629566839 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f09480287b0 msgr2=0x7f094802ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:13.526 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.523+0000 7f096f711700 1 --2- 172.21.15.170:0/1629566839 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f09480287b0 0x7f094802ac70 secure :-1 s=READY pgs=21 cs=0 l=1 rx=0x7f095c000ff0 tx=0x7f095c0016c0).stop 2020-03-19T21:18:13.526 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.523+0000 7f096f711700 1 -- 172.21.15.170:0/1629566839 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0968125840 msgr2=0x7f09680664e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:13.526 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.523+0000 7f096f711700 1 --2- 172.21.15.170:0/1629566839 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0968125840 0x7f09680664e0 secure :-1 s=READY pgs=85 cs=0 l=1 rx=0x7f0964002f30 tx=0x7f0964007a40).stop 2020-03-19T21:18:13.526 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.524+0000 7f096f711700 1 -- 172.21.15.170:0/1629566839 shutdown_connections 2020-03-19T21:18:13.527 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.524+0000 7f096f711700 1 --2- 172.21.15.170:0/1629566839 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f09480287b0 0x7f094802ac70 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:13.527 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.524+0000 7f096f711700 1 --2- 172.21.15.170:0/1629566839 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0968125840 0x7f09680664e0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:13.527 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.524+0000 7f096f711700 1 -- 172.21.15.170:0/1629566839 >> 172.21.15.170:0/1629566839 conn(0x7f0968091390 msgr2=0x7f0968060110 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:13.527 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.525+0000 7f096f711700 1 -- 172.21.15.170:0/1629566839 shutdown_connections 2020-03-19T21:18:13.527 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.526+0000 7f096f711700 1 -- 172.21.15.170:0/1629566839 wait complete. 2020-03-19T21:18:13.528 INFO:teuthology.orchestra.run.smithi170.stderr:new fs with metadata pool 2 and data pool 3 2020-03-19T21:18:13.538 INFO:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-03-19T21:18:13.643 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.641+0000 7f9dc2da7700 1 Processor -- start 2020-03-19T21:18:13.643 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.641+0000 7f9dc2da7700 1 -- start start 2020-03-19T21:18:13.649 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.641+0000 7f9dc2da7700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f9dbc126dc0 0x7f9dbc1291b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T21:18:13.650 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.641+0000 7f9dc2da7700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9dbc062590 con 0x7f9dbc0667d0 2020-03-19T21:18:13.650 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.641+0000 7f9dc2da7700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9dbc0907d0 con 0x7f9dbc126dc0 2020-03-19T21:18:13.650 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.641+0000 7f9dbbfff700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f9dbc126dc0 0x7f9dbc1291b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:13.650 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.641+0000 7f9dc0b43700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f9dbc0667d0 0x7f9dbc066bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:53604/0 (socket says 172.21.15.170:53604) 2020-03-19T21:18:13.650 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.641+0000 7f9dc0b43700 1 -- 172.21.15.170:0/1192211271 learned_addr learned my addr 172.21.15.170:0/1192211271 (peer_addr_for_me v1:172.21.15.170:0/0) 2020-03-19T21:18:13.650 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.641+0000 7f9dbbfff700 1 -- 172.21.15.170:0/1192211271 >> v1:172.21.15.170:6789/0 conn(0x7f9dbc0667d0 legacy=0x7f9dbc066bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:13.651 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.642+0000 7f9dbbfff700 1 -- 172.21.15.170:0/1192211271 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dbc08d290 con 0x7f9dbc126dc0 2020-03-19T21:18:13.651 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.641+0000 7f9dbb7fe700 1 -- 172.21.15.170:0/1192211271 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1167009383 0 0) 0x7f9dbc062590 con 0x7f9dbc0667d0 2020-03-19T21:18:13.651 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.642+0000 7f9dbbfff700 1 --2- 172.21.15.170:0/1192211271 >> v2:172.21.15.170:3300/0 conn(0x7f9dbc126dc0 0x7f9dbc1291b0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f9db0002f30 tx=0x7f9db0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:13.652 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.642+0000 7f9dbbfff700 1 --2- 172.21.15.170:0/1192211271 >> v2:172.21.15.170:3300/0 conn(0x7f9dbc126dc0 0x7f9dbc1291b0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f9db0002f30 tx=0x7f9db0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:13.652 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.642+0000 7f9dbbfff700 1 --2- 172.21.15.170:0/1192211271 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9dbc126dc0 0x7f9dbc1291b0 secure :-1 s=READY pgs=87 cs=0 l=1 rx=0x7f9db0002f30 tx=0x7f9db0007a40).ready entity=mon.0 client_cookie=3c2fd1f0de41c3b3 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:13.652 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.642+0000 7f9dbbfff700 1 --2- 172.21.15.170:0/1192211271 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9dbc126dc0 0x7f9dbc1291b0 secure :-1 s=THROTTLE_DONE pgs=87 cs=0 l=1 rx=0x7f9db0002f30 tx=0x7f9db0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:13.652 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.642+0000 7f9dbb7fe700 1 -- 172.21.15.170:0/1192211271 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f9db0008e80 con 0x7f9dbc126dc0 2020-03-19T21:18:13.652 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.642+0000 7f9dbbfff700 1 --2- 172.21.15.170:0/1192211271 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9dbc126dc0 0x7f9dbc1291b0 secure :-1 s=THROTTLE_DONE pgs=87 cs=0 l=1 rx=0x7f9db0002f30 tx=0x7f9db0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:13.652 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.642+0000 7f9dbb7fe700 1 -- 172.21.15.170:0/1192211271 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9db00113f0 con 0x7f9dbc126dc0 2020-03-19T21:18:13.652 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.642+0000 7f9dbbfff700 1 --2- 172.21.15.170:0/1192211271 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9dbc126dc0 0x7f9dbc1291b0 secure :-1 s=THROTTLE_DONE pgs=87 cs=0 l=1 rx=0x7f9db0002f30 tx=0x7f9db0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:13.653 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.642+0000 7f9dbb7fe700 1 -- 172.21.15.170:0/1192211271 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f9db0011690 con 0x7f9dbc126dc0 2020-03-19T21:18:13.653 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.642+0000 7f9dc2da7700 1 -- 172.21.15.170:0/1192211271 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9dbc126dc0 msgr2=0x7f9dbc1291b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:13.653 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.642+0000 7f9dc2da7700 1 --2- 172.21.15.170:0/1192211271 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9dbc126dc0 0x7f9dbc1291b0 secure :-1 s=READY pgs=87 cs=0 l=1 rx=0x7f9db0002f30 tx=0x7f9db0007a40).stop 2020-03-19T21:18:13.653 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.642+0000 7f9dc2da7700 1 -- 172.21.15.170:0/1192211271 shutdown_connections 2020-03-19T21:18:13.653 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.642+0000 7f9dc2da7700 1 --2- 172.21.15.170:0/1192211271 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9dbc126dc0 0x7f9dbc1291b0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:13.653 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.642+0000 7f9dc2da7700 1 -- 172.21.15.170:0/1192211271 >> 172.21.15.170:0/1192211271 conn(0x7f9dbc091540 msgr2=0x7f9dbc091950 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:13.653 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.642+0000 7f9dc2da7700 1 -- 172.21.15.170:0/1192211271 shutdown_connections 2020-03-19T21:18:13.654 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.642+0000 7f9dc2da7700 1 -- 172.21.15.170:0/1192211271 wait complete. 2020-03-19T21:18:13.654 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.643+0000 7f9dc2da7700 1 Processor -- start 2020-03-19T21:18:13.654 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.643+0000 7f9dc2da7700 1 -- start start 2020-03-19T21:18:13.654 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.644+0000 7f9dc2da7700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f9dbc126dc0 0x7f9dbc135260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:13.654 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.644+0000 7f9dc2da7700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9dbc11bd60 con 0x7f9dbc0667d0 2020-03-19T21:18:13.654 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.644+0000 7f9dc2da7700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9dbc11e480 con 0x7f9dbc126dc0 2020-03-19T21:18:13.654 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.644+0000 7f9dbbfff700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f9dbc126dc0 0x7f9dbc135260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:13.655 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.644+0000 7f9dc0b43700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f9dbc0667d0 0x7f9dbc1245a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:53608/0 (socket says 172.21.15.170:53608) 2020-03-19T21:18:13.655 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.644+0000 7f9dc0b43700 1 -- 172.21.15.170:0/31939832 learned_addr learned my addr 172.21.15.170:0/31939832 (peer_addr_for_me v1:172.21.15.170:0/0) 2020-03-19T21:18:13.655 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.644+0000 7f9dbbfff700 1 -- 172.21.15.170:0/31939832 >> v1:172.21.15.170:6789/0 conn(0x7f9dbc0667d0 legacy=0x7f9dbc1245a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:13.655 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.644+0000 7f9dbbfff700 1 -- 172.21.15.170:0/31939832 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dbc11cb40 con 0x7f9dbc126dc0 2020-03-19T21:18:13.655 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.644+0000 7f9db9ffb700 1 -- 172.21.15.170:0/31939832 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3511168230 0 0) 0x7f9dbc11bd60 con 0x7f9dbc0667d0 2020-03-19T21:18:13.655 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.644+0000 7f9dbbfff700 1 --2- 172.21.15.170:0/31939832 >> v2:172.21.15.170:3300/0 conn(0x7f9dbc126dc0 0x7f9dbc135260 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f9db0002f30 tx=0x7f9db0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:13.656 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.644+0000 7f9dbbfff700 1 --2- 172.21.15.170:0/31939832 >> v2:172.21.15.170:3300/0 conn(0x7f9dbc126dc0 0x7f9dbc135260 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f9db0002f30 tx=0x7f9db0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:13.656 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.644+0000 7f9dbbfff700 1 --2- 172.21.15.170:0/31939832 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9dbc126dc0 0x7f9dbc135260 secure :-1 s=READY pgs=89 cs=0 l=1 rx=0x7f9db0002f30 tx=0x7f9db0007a40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:13.656 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.644+0000 7f9dbbfff700 1 --2- 172.21.15.170:0/31939832 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9dbc126dc0 0x7f9dbc135260 secure :-1 s=THROTTLE_DONE pgs=89 cs=0 l=1 rx=0x7f9db0002f30 tx=0x7f9db0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:13.656 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.644+0000 7f9db9ffb700 1 -- 172.21.15.170:0/31939832 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f9db0011e70 con 0x7f9dbc126dc0 2020-03-19T21:18:13.656 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.644+0000 7f9dbbfff700 1 --2- 172.21.15.170:0/31939832 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9dbc126dc0 0x7f9dbc135260 secure :-1 s=THROTTLE_DONE pgs=89 cs=0 l=1 rx=0x7f9db0002f30 tx=0x7f9db0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:13.656 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.644+0000 7f9dbbfff700 1 --2- 172.21.15.170:0/31939832 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9dbc126dc0 0x7f9dbc135260 secure :-1 s=THROTTLE_DONE pgs=89 cs=0 l=1 rx=0x7f9db0002f30 tx=0x7f9db0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:13.656 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.644+0000 7f9db9ffb700 1 -- 172.21.15.170:0/31939832 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9db0007150 con 0x7f9dbc126dc0 2020-03-19T21:18:13.656 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.644+0000 7f9db9ffb700 1 -- 172.21.15.170:0/31939832 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f9db00073f0 con 0x7f9dbc126dc0 2020-03-19T21:18:13.657 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.644+0000 7f9dc2da7700 1 -- 172.21.15.170:0/31939832 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dbc11ba90 con 0x7f9dbc126dc0 2020-03-19T21:18:13.657 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.644+0000 7f9dc2da7700 1 -- 172.21.15.170:0/31939832 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dbc11ce30 con 0x7f9dbc126dc0 2020-03-19T21:18:13.657 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.645+0000 7f9dbbfff700 1 --2- 172.21.15.170:0/31939832 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9dbc126dc0 0x7f9dbc135260 secure :-1 s=THROTTLE_DONE pgs=89 cs=0 l=1 rx=0x7f9db0002f30 tx=0x7f9db0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:13.657 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.645+0000 7f9db9ffb700 1 -- 172.21.15.170:0/31939832 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 4) v1 ==== 29603+0+0 (secure 0 0 0) 0x7f9db00056c0 con 0x7f9dbc126dc0 2020-03-19T21:18:13.657 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.645+0000 7f9dbbfff700 1 --2- 172.21.15.170:0/31939832 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9dbc126dc0 0x7f9dbc135260 secure :-1 s=THROTTLE_DONE pgs=89 cs=0 l=1 rx=0x7f9db0002f30 tx=0x7f9db0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:13.657 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.645+0000 7f9db9ffb700 1 --2- 172.21.15.170:0/31939832 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f9d9c028280 0x7f9d9c02a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:13.658 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.645+0000 7f9db9ffb700 1 -- 172.21.15.170:0/31939832 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3403+0+0 (secure 0 0 0) 0x7f9db00348e0 con 0x7f9dbc126dc0 2020-03-19T21:18:13.658 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.645+0000 7f9dc0b43700 1 --2- 172.21.15.170:0/31939832 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f9d9c028280 0x7f9d9c02a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:13.658 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.645+0000 7f9dc2da7700 1 -- 172.21.15.170:0/31939832 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9dbc056db0 con 0x7f9dbc126dc0 2020-03-19T21:18:13.658 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.645+0000 7f9dc0b43700 1 --2- 172.21.15.170:0/31939832 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f9d9c028280 0x7f9d9c02a740 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f9dac005560 tx=0x7f9dac0016c0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:13.658 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.645+0000 7f9dc0b43700 1 --2- 172.21.15.170:0/31939832 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f9d9c028280 0x7f9d9c02a740 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f9dac005560 tx=0x7f9dac0016c0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:13.658 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.645+0000 7f9dc0b43700 1 --2- 172.21.15.170:0/31939832 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f9d9c028280 0x7f9d9c02a740 secure :-1 s=READY pgs=22 cs=0 l=1 rx=0x7f9dac005560 tx=0x7f9dac0016c0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:13.659 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.649+0000 7f9dbbfff700 1 --2- 172.21.15.170:0/31939832 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9dbc126dc0 0x7f9dbc135260 secure :-1 s=THROTTLE_DONE pgs=89 cs=0 l=1 rx=0x7f9db0002f30 tx=0x7f9db0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:13.659 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.649+0000 7f9db9ffb700 1 -- 172.21.15.170:0/31939832 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+126772 (secure 0 0 0) 0x7f9db0009e30 con 0x7f9dbc126dc0 2020-03-19T21:18:13.791 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.789+0000 7f9dc2da7700 1 -- 172.21.15.170:0/31939832 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9dbc067850 con 0x7f9dbc126dc0 2020-03-19T21:18:13.792 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.790+0000 7f9dbbfff700 1 --2- 172.21.15.170:0/31939832 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9dbc126dc0 0x7f9dbc135260 secure :-1 s=THROTTLE_DONE pgs=89 cs=0 l=1 rx=0x7f9db0002f30 tx=0x7f9db0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:13.792 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.790+0000 7f9db9ffb700 1 -- 172.21.15.170:0/31939832 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v15) v1 ==== 74+0+8850 (secure 0 0 0) 0x7f9db0009e30 con 0x7f9dbc126dc0 2020-03-19T21:18:13.793 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:18:13.793 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":15,"fsid":"12d0a33b-6b40-4398-90b4-6b5df1da388e","created":"2020-03-19T21:17:59.135747+0000","modified":"2020-03-19T21:18:13.514432+0000","last_up_change":"2020-03-19T21:18:05.158515+0000","last_in_change":"2020-03-19T21:18:05.158515+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":3,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2020-03-19T21:18:08.391604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2020-03-19T21:18:11.286536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2020-03-19T21:18:12.597110+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"32ca3517-1077-4102-9aa4-edfcebe8e3d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":20680},{"type":"v1","addr":"172.21.15.170:6801","nonce":20680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":20680},{"type":"v1","addr":"172.21.15.170:6803","nonce":20680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":20680},{"type":"v1","addr":"172.21.15.170:6807","nonce":20680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":20680},{"type":"v1","addr":"172.21.15.170:6805","nonce":20680}]},"public_addr":"172.21.15.170:6801/20680","cluster_addr":"172.21.15.170:6803/20680","heartbeat_back_addr":"172.21.15.170:6807/20680","heartbeat_front_addr":"172.21.15.170:6805/20680","state":["exists","up"]},{"osd":1,"uuid":"470df64b-60bc-4f5b-9b53-64495cd7c5e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":20681},{"type":"v1","addr":"172.21.15.170:6809","nonce":20681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":20681},{"type":"v1","addr":"172.21.15.170:6811","nonce":20681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":20681},{"type":"v1","addr":"172.21.15.170:6815","nonce":20681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":20681},{"type":"v1","addr":"172.21.15.170:6813","nonce":20681}]},"public_addr":"172.21.15.170:6809/20681","cluster_addr":"172.21.15.170:6811/20681","heartbeat_back_addr":"172.21.15.170:6815/20681","heartbeat_front_addr":"172.21.15.170:6813/20681","state":["exists","up"]},{"osd":2,"uuid":"ccfb9d82-b759-4ff4-8ddf-4f1d741bac5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":20682},{"type":"v1","addr":"172.21.15.170:6817","nonce":20682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":20682},{"type":"v1","addr":"172.21.15.170:6819","nonce":20682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":20682},{"type":"v1","addr":"172.21.15.170:6823","nonce":20682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":20682},{"type":"v1","addr":"172.21.15.170:6821","nonce":20682}]},"public_addr":"172.21.15.170:6817/20682","cluster_addr":"172.21.15.170:6819/20682","heartbeat_back_addr":"172.21.15.170:6823/20682","heartbeat_front_addr":"172.21.15.170:6821/20682","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2020-03-19T21:18:03.177293+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2020-03-19T21:18:03.263179+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2020-03-19T21:18:03.183223+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-03-19T21:18:13.794 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.791+0000 7f9dc2da7700 1 -- 172.21.15.170:0/31939832 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f9d9c028280 msgr2=0x7f9d9c02a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:13.794 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.791+0000 7f9dc2da7700 1 --2- 172.21.15.170:0/31939832 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f9d9c028280 0x7f9d9c02a740 secure :-1 s=READY pgs=22 cs=0 l=1 rx=0x7f9dac005560 tx=0x7f9dac0016c0).stop 2020-03-19T21:18:13.794 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.791+0000 7f9dc2da7700 1 -- 172.21.15.170:0/31939832 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9dbc126dc0 msgr2=0x7f9dbc135260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:13.794 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.791+0000 7f9dc2da7700 1 --2- 172.21.15.170:0/31939832 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9dbc126dc0 0x7f9dbc135260 secure :-1 s=READY pgs=89 cs=0 l=1 rx=0x7f9db0002f30 tx=0x7f9db0007a40).stop 2020-03-19T21:18:13.794 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.791+0000 7f9dc2da7700 1 -- 172.21.15.170:0/31939832 shutdown_connections 2020-03-19T21:18:13.795 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.791+0000 7f9dc2da7700 1 --2- 172.21.15.170:0/31939832 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f9d9c028280 0x7f9d9c02a740 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:13.795 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.791+0000 7f9dc2da7700 1 --2- 172.21.15.170:0/31939832 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9dbc126dc0 0x7f9dbc135260 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:13.795 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.791+0000 7f9dc2da7700 1 -- 172.21.15.170:0/31939832 >> 172.21.15.170:0/31939832 conn(0x7f9dbc091540 msgr2=0x7f9dbc125910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:13.795 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.791+0000 7f9dc2da7700 1 -- 172.21.15.170:0/31939832 shutdown_connections 2020-03-19T21:18:13.795 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.791+0000 7f9dc2da7700 1 -- 172.21.15.170:0/31939832 wait complete. 2020-03-19T21:18:13.804 INFO:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-03-19T21:18:13.909 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.906+0000 7f5a35d6c700 1 Processor -- start 2020-03-19T21:18:13.909 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.906+0000 7f5a35d6c700 1 -- start start 2020-03-19T21:18:13.915 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.907+0000 7f5a35d6c700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f5a300907c0 0x7f5a3008daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T21:18:13.915 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.907+0000 7f5a35d6c700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a300605e0 con 0x7f5a3008d000 2020-03-19T21:18:13.915 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.907+0000 7f5a35d6c700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a3008ef70 con 0x7f5a300907c0 2020-03-19T21:18:13.915 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.907+0000 7f5a2effd700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f5a300907c0 0x7f5a3008daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:13.915 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.907+0000 7f5a2f7fe700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f5a3008d000 0x7f5a3008d3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:53614/0 (socket says 172.21.15.170:53614) 2020-03-19T21:18:13.915 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.907+0000 7f5a2f7fe700 1 -- 172.21.15.170:0/1311584275 learned_addr learned my addr 172.21.15.170:0/1311584275 (peer_addr_for_me v1:172.21.15.170:0/0) 2020-03-19T21:18:13.916 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.907+0000 7f5a2effd700 1 -- 172.21.15.170:0/1311584275 >> v1:172.21.15.170:6789/0 conn(0x7f5a3008d000 legacy=0x7f5a3008d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:13.916 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.907+0000 7f5a2effd700 1 -- 172.21.15.170:0/1311584275 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a30060810 con 0x7f5a300907c0 2020-03-19T21:18:13.916 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.907+0000 7f5a2e7fc700 1 -- 172.21.15.170:0/1311584275 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1148919900 0 0) 0x7f5a300605e0 con 0x7f5a3008d000 2020-03-19T21:18:13.916 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.907+0000 7f5a2effd700 1 --2- 172.21.15.170:0/1311584275 >> v2:172.21.15.170:3300/0 conn(0x7f5a300907c0 0x7f5a3008daf0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f5a30090c70 tx=0x7f5a240088e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:13.916 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.907+0000 7f5a2effd700 1 --2- 172.21.15.170:0/1311584275 >> v2:172.21.15.170:3300/0 conn(0x7f5a300907c0 0x7f5a3008daf0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f5a30090c70 tx=0x7f5a240088e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:13.916 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.908+0000 7f5a2effd700 1 --2- 172.21.15.170:0/1311584275 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5a300907c0 0x7f5a3008daf0 secure :-1 s=READY pgs=91 cs=0 l=1 rx=0x7f5a30090c70 tx=0x7f5a240088e0).ready entity=mon.0 client_cookie=5e4fa4f60470f88 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:13.916 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.908+0000 7f5a2effd700 1 --2- 172.21.15.170:0/1311584275 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5a300907c0 0x7f5a3008daf0 secure :-1 s=THROTTLE_DONE pgs=91 cs=0 l=1 rx=0x7f5a30090c70 tx=0x7f5a240088e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:13.917 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.908+0000 7f5a2e7fc700 1 -- 172.21.15.170:0/1311584275 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f5a240113f0 con 0x7f5a300907c0 2020-03-19T21:18:13.917 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.908+0000 7f5a2effd700 1 --2- 172.21.15.170:0/1311584275 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5a300907c0 0x7f5a3008daf0 secure :-1 s=THROTTLE_DONE pgs=91 cs=0 l=1 rx=0x7f5a30090c70 tx=0x7f5a240088e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:13.917 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.908+0000 7f5a2e7fc700 1 -- 172.21.15.170:0/1311584275 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a24011550 con 0x7f5a300907c0 2020-03-19T21:18:13.917 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.908+0000 7f5a2effd700 1 --2- 172.21.15.170:0/1311584275 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5a300907c0 0x7f5a3008daf0 secure :-1 s=THROTTLE_DONE pgs=91 cs=0 l=1 rx=0x7f5a30090c70 tx=0x7f5a240088e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:13.917 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.908+0000 7f5a2e7fc700 1 -- 172.21.15.170:0/1311584275 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f5a24011830 con 0x7f5a300907c0 2020-03-19T21:18:13.917 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.908+0000 7f5a35d6c700 1 -- 172.21.15.170:0/1311584275 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5a300907c0 msgr2=0x7f5a3008daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:13.917 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.908+0000 7f5a35d6c700 1 --2- 172.21.15.170:0/1311584275 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5a300907c0 0x7f5a3008daf0 secure :-1 s=READY pgs=91 cs=0 l=1 rx=0x7f5a30090c70 tx=0x7f5a240088e0).stop 2020-03-19T21:18:13.918 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.908+0000 7f5a35d6c700 1 -- 172.21.15.170:0/1311584275 shutdown_connections 2020-03-19T21:18:13.918 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.908+0000 7f5a35d6c700 1 --2- 172.21.15.170:0/1311584275 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5a300907c0 0x7f5a3008daf0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:13.918 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.908+0000 7f5a35d6c700 1 -- 172.21.15.170:0/1311584275 >> 172.21.15.170:0/1311584275 conn(0x7f5a30091540 msgr2=0x7f5a30091950 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:13.918 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.908+0000 7f5a35d6c700 1 -- 172.21.15.170:0/1311584275 shutdown_connections 2020-03-19T21:18:13.918 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.908+0000 7f5a35d6c700 1 -- 172.21.15.170:0/1311584275 wait complete. 2020-03-19T21:18:13.919 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.909+0000 7f5a35d6c700 1 Processor -- start 2020-03-19T21:18:13.919 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.909+0000 7f5a35d6c700 1 -- start start 2020-03-19T21:18:13.919 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.909+0000 7f5a35d6c700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f5a300907c0 0x7f5a301228a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:13.919 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.909+0000 7f5a35d6c700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a3011ff70 con 0x7f5a3008d000 2020-03-19T21:18:13.919 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.909+0000 7f5a35d6c700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a3011fc60 con 0x7f5a300907c0 2020-03-19T21:18:13.919 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.910+0000 7f5a2effd700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f5a300907c0 0x7f5a301228a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:13.919 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.910+0000 7f5a2f7fe700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f5a3008d000 0x7f5a30122190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:53618/0 (socket says 172.21.15.170:53618) 2020-03-19T21:18:13.920 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.910+0000 7f5a2f7fe700 1 -- 172.21.15.170:0/924487626 learned_addr learned my addr 172.21.15.170:0/924487626 (peer_addr_for_me v1:172.21.15.170:0/0) 2020-03-19T21:18:13.920 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.910+0000 7f5a2effd700 1 -- 172.21.15.170:0/924487626 >> v1:172.21.15.170:6789/0 conn(0x7f5a3008d000 legacy=0x7f5a30122190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:13.920 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.910+0000 7f5a2cff9700 1 -- 172.21.15.170:0/924487626 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2398470064 0 0) 0x7f5a3011ff70 con 0x7f5a3008d000 2020-03-19T21:18:13.920 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.910+0000 7f5a2effd700 1 -- 172.21.15.170:0/924487626 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a3011d270 con 0x7f5a300907c0 2020-03-19T21:18:13.920 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.910+0000 7f5a2effd700 1 --2- 172.21.15.170:0/924487626 >> v2:172.21.15.170:3300/0 conn(0x7f5a300907c0 0x7f5a301228a0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f5a30090c70 tx=0x7f5a30067850).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:13.920 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.910+0000 7f5a2effd700 1 --2- 172.21.15.170:0/924487626 >> v2:172.21.15.170:3300/0 conn(0x7f5a300907c0 0x7f5a301228a0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f5a30090c70 tx=0x7f5a30067850).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:13.920 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.910+0000 7f5a2effd700 1 --2- 172.21.15.170:0/924487626 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5a300907c0 0x7f5a301228a0 secure :-1 s=READY pgs=93 cs=0 l=1 rx=0x7f5a30090c70 tx=0x7f5a30067850).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:13.921 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.910+0000 7f5a2effd700 1 --2- 172.21.15.170:0/924487626 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5a300907c0 0x7f5a301228a0 secure :-1 s=THROTTLE_DONE pgs=93 cs=0 l=1 rx=0x7f5a30090c70 tx=0x7f5a30067850).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:13.921 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.910+0000 7f5a2cff9700 1 -- 172.21.15.170:0/924487626 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f5a2400f740 con 0x7f5a300907c0 2020-03-19T21:18:13.921 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.910+0000 7f5a2effd700 1 --2- 172.21.15.170:0/924487626 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5a300907c0 0x7f5a301228a0 secure :-1 s=THROTTLE_DONE pgs=93 cs=0 l=1 rx=0x7f5a30090c70 tx=0x7f5a30067850).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:13.921 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.910+0000 7f5a2cff9700 1 -- 172.21.15.170:0/924487626 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a240056c0 con 0x7f5a300907c0 2020-03-19T21:18:13.921 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.910+0000 7f5a2effd700 1 --2- 172.21.15.170:0/924487626 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5a300907c0 0x7f5a301228a0 secure :-1 s=THROTTLE_DONE pgs=93 cs=0 l=1 rx=0x7f5a30090c70 tx=0x7f5a30067850).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:13.921 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.910+0000 7f5a35d6c700 1 -- 172.21.15.170:0/924487626 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a3011f340 con 0x7f5a300907c0 2020-03-19T21:18:13.921 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.910+0000 7f5a2cff9700 1 -- 172.21.15.170:0/924487626 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f5a240059a0 con 0x7f5a300907c0 2020-03-19T21:18:13.922 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.910+0000 7f5a35d6c700 1 -- 172.21.15.170:0/924487626 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a3011e8f0 con 0x7f5a300907c0 2020-03-19T21:18:13.922 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.910+0000 7f5a2effd700 1 --2- 172.21.15.170:0/924487626 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5a300907c0 0x7f5a301228a0 secure :-1 s=THROTTLE_DONE pgs=93 cs=0 l=1 rx=0x7f5a30090c70 tx=0x7f5a30067850).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:13.922 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.911+0000 7f5a2cff9700 1 -- 172.21.15.170:0/924487626 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 4) v1 ==== 29603+0+0 (secure 0 0 0) 0x7f5a24005ba0 con 0x7f5a300907c0 2020-03-19T21:18:13.922 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.911+0000 7f5a2effd700 1 --2- 172.21.15.170:0/924487626 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5a300907c0 0x7f5a301228a0 secure :-1 s=THROTTLE_DONE pgs=93 cs=0 l=1 rx=0x7f5a30090c70 tx=0x7f5a30067850).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:13.922 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.911+0000 7f5a2cff9700 1 --2- 172.21.15.170:0/924487626 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f5a10028200 0x7f5a1002a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:13.922 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.911+0000 7f5a2cff9700 1 -- 172.21.15.170:0/924487626 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3403+0+0 (secure 0 0 0) 0x7f5a240347c0 con 0x7f5a300907c0 2020-03-19T21:18:13.922 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.911+0000 7f5a2f7fe700 1 --2- 172.21.15.170:0/924487626 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f5a10028200 0x7f5a1002a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:13.923 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.911+0000 7f5a2f7fe700 1 --2- 172.21.15.170:0/924487626 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f5a10028200 0x7f5a1002a6c0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f5a20005560 tx=0x7f5a200016c0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:13.923 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.911+0000 7f5a35d6c700 1 -- 172.21.15.170:0/924487626 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a3011fdd0 con 0x7f5a300907c0 2020-03-19T21:18:13.923 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.911+0000 7f5a2f7fe700 1 --2- 172.21.15.170:0/924487626 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f5a10028200 0x7f5a1002a6c0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f5a20005560 tx=0x7f5a200016c0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:13.923 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.911+0000 7f5a2f7fe700 1 --2- 172.21.15.170:0/924487626 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f5a10028200 0x7f5a1002a6c0 secure :-1 s=READY pgs=23 cs=0 l=1 rx=0x7f5a20005560 tx=0x7f5a200016c0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:13.924 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.915+0000 7f5a2effd700 1 --2- 172.21.15.170:0/924487626 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5a300907c0 0x7f5a301228a0 secure :-1 s=THROTTLE_DONE pgs=93 cs=0 l=1 rx=0x7f5a30090c70 tx=0x7f5a30067850).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:13.924 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:13.915+0000 7f5a2cff9700 1 -- 172.21.15.170:0/924487626 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+126772 (secure 0 0 0) 0x7f5a3011fdd0 con 0x7f5a300907c0 2020-03-19T21:18:14.055 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:14.053+0000 7f5a35d6c700 1 -- 172.21.15.170:0/924487626 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5a3011fdd0 con 0x7f5a300907c0 2020-03-19T21:18:14.056 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:14.053+0000 7f5a2effd700 1 --2- 172.21.15.170:0/924487626 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5a300907c0 0x7f5a301228a0 secure :-1 s=THROTTLE_DONE pgs=93 cs=0 l=1 rx=0x7f5a30090c70 tx=0x7f5a30067850).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:14.056 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:14.054+0000 7f5a2cff9700 1 -- 172.21.15.170:0/924487626 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v15) v1 ==== 74+0+8850 (secure 0 0 0) 0x7f5a3011fdd0 con 0x7f5a300907c0 2020-03-19T21:18:14.056 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:18:14.056 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":15,"fsid":"12d0a33b-6b40-4398-90b4-6b5df1da388e","created":"2020-03-19T21:17:59.135747+0000","modified":"2020-03-19T21:18:13.514432+0000","last_up_change":"2020-03-19T21:18:05.158515+0000","last_in_change":"2020-03-19T21:18:05.158515+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":3,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2020-03-19T21:18:08.391604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2020-03-19T21:18:11.286536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2020-03-19T21:18:12.597110+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"32ca3517-1077-4102-9aa4-edfcebe8e3d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":20680},{"type":"v1","addr":"172.21.15.170:6801","nonce":20680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":20680},{"type":"v1","addr":"172.21.15.170:6803","nonce":20680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":20680},{"type":"v1","addr":"172.21.15.170:6807","nonce":20680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":20680},{"type":"v1","addr":"172.21.15.170:6805","nonce":20680}]},"public_addr":"172.21.15.170:6801/20680","cluster_addr":"172.21.15.170:6803/20680","heartbeat_back_addr":"172.21.15.170:6807/20680","heartbeat_front_addr":"172.21.15.170:6805/20680","state":["exists","up"]},{"osd":1,"uuid":"470df64b-60bc-4f5b-9b53-64495cd7c5e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":20681},{"type":"v1","addr":"172.21.15.170:6809","nonce":20681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":20681},{"type":"v1","addr":"172.21.15.170:6811","nonce":20681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":20681},{"type":"v1","addr":"172.21.15.170:6815","nonce":20681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":20681},{"type":"v1","addr":"172.21.15.170:6813","nonce":20681}]},"public_addr":"172.21.15.170:6809/20681","cluster_addr":"172.21.15.170:6811/20681","heartbeat_back_addr":"172.21.15.170:6815/20681","heartbeat_front_addr":"172.21.15.170:6813/20681","state":["exists","up"]},{"osd":2,"uuid":"ccfb9d82-b759-4ff4-8ddf-4f1d741bac5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":20682},{"type":"v1","addr":"172.21.15.170:6817","nonce":20682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":20682},{"type":"v1","addr":"172.21.15.170:6819","nonce":20682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":20682},{"type":"v1","addr":"172.21.15.170:6823","nonce":20682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":20682},{"type":"v1","addr":"172.21.15.170:6821","nonce":20682}]},"public_addr":"172.21.15.170:6817/20682","cluster_addr":"172.21.15.170:6819/20682","heartbeat_back_addr":"172.21.15.170:6823/20682","heartbeat_front_addr":"172.21.15.170:6821/20682","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2020-03-19T21:18:03.177293+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2020-03-19T21:18:03.263179+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2020-03-19T21:18:03.183223+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-03-19T21:18:14.057 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:14.055+0000 7f5a35d6c700 1 -- 172.21.15.170:0/924487626 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f5a10028200 msgr2=0x7f5a1002a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:14.058 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:14.055+0000 7f5a35d6c700 1 --2- 172.21.15.170:0/924487626 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f5a10028200 0x7f5a1002a6c0 secure :-1 s=READY pgs=23 cs=0 l=1 rx=0x7f5a20005560 tx=0x7f5a200016c0).stop 2020-03-19T21:18:14.058 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:14.055+0000 7f5a35d6c700 1 -- 172.21.15.170:0/924487626 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5a300907c0 msgr2=0x7f5a301228a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:14.058 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:14.055+0000 7f5a35d6c700 1 --2- 172.21.15.170:0/924487626 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5a300907c0 0x7f5a301228a0 secure :-1 s=READY pgs=93 cs=0 l=1 rx=0x7f5a30090c70 tx=0x7f5a30067850).stop 2020-03-19T21:18:14.058 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:14.055+0000 7f5a35d6c700 1 -- 172.21.15.170:0/924487626 shutdown_connections 2020-03-19T21:18:14.058 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:14.055+0000 7f5a35d6c700 1 --2- 172.21.15.170:0/924487626 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f5a10028200 0x7f5a1002a6c0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:14.058 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:14.055+0000 7f5a35d6c700 1 --2- 172.21.15.170:0/924487626 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5a300907c0 0x7f5a301228a0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:14.058 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:14.055+0000 7f5a35d6c700 1 -- 172.21.15.170:0/924487626 >> 172.21.15.170:0/924487626 conn(0x7f5a30091540 msgr2=0x7f5a300629b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:14.059 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:14.055+0000 7f5a35d6c700 1 -- 172.21.15.170:0/924487626 shutdown_connections 2020-03-19T21:18:14.059 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:14.055+0000 7f5a35d6c700 1 -- 172.21.15.170:0/924487626 wait complete. 2020-03-19T21:18:14.068 INFO:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs set cephfs standby_count_wanted 0 2020-03-19T21:18:14.175 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:14.172+0000 7fcb387ad700 1 Processor -- start 2020-03-19T21:18:14.175 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:14.172+0000 7fcb387ad700 1 -- start start 2020-03-19T21:18:14.181 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:14.173+0000 7fcb387ad700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fcb301224c0 0x7fcb30122ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T21:18:14.181 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:14.173+0000 7fcb387ad700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb301228a0 con 0x7fcb30122f40 2020-03-19T21:18:14.181 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:14.173+0000 7fcb387ad700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb30092200 con 0x7fcb301224c0 2020-03-19T21:18:14.181 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:14.173+0000 7fcb36549700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fcb301224c0 0x7fcb30122ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:14.181 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:14.173+0000 7fcb35d48700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fcb30122f40 0x7fcb30060110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:53626/0 (socket says 172.21.15.170:53626) 2020-03-19T21:18:14.181 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:14.173+0000 7fcb35d48700 1 -- 172.21.15.170:0/3178593439 learned_addr learned my addr 172.21.15.170:0/3178593439 (peer_addr_for_me v1:172.21.15.170:0/0) 2020-03-19T21:18:14.182 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:14.173+0000 7fcb36549700 1 -- 172.21.15.170:0/3178593439 >> v1:172.21.15.170:6789/0 conn(0x7fcb30122f40 legacy=0x7fcb30060110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:14.182 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:14.173+0000 7fcb36549700 1 -- 172.21.15.170:0/3178593439 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb30090b60 con 0x7fcb301224c0 2020-03-19T21:18:14.182 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:14.173+0000 7fcb35547700 1 -- 172.21.15.170:0/3178593439 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1111513158 0 0) 0x7fcb2c0010d0 con 0x7fcb30122f40 2020-03-19T21:18:14.182 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:14.174+0000 7fcb36549700 1 --2- 172.21.15.170:0/3178593439 >> v2:172.21.15.170:3300/0 conn(0x7fcb301224c0 0x7fcb30122ae0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7fcb20002f30 tx=0x7fcb20007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:14.182 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:14.174+0000 7fcb36549700 1 --2- 172.21.15.170:0/3178593439 >> v2:172.21.15.170:3300/0 conn(0x7fcb301224c0 0x7fcb30122ae0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7fcb20002f30 tx=0x7fcb20007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:14.182 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:14.174+0000 7fcb36549700 1 --2- 172.21.15.170:0/3178593439 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb301224c0 0x7fcb30122ae0 secure :-1 s=READY pgs=95 cs=0 l=1 rx=0x7fcb20002f30 tx=0x7fcb20007a40).ready entity=mon.0 client_cookie=1464724e4b878cc3 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:14.182 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:14.174+0000 7fcb36549700 1 --2- 172.21.15.170:0/3178593439 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb301224c0 0x7fcb30122ae0 secure :-1 s=THROTTLE_DONE pgs=95 cs=0 l=1 rx=0x7fcb20002f30 tx=0x7fcb20007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:14.182 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:14.174+0000 7fcb35547700 1 -- 172.21.15.170:0/3178593439 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fcb20008e80 con 0x7fcb301224c0 2020-03-19T21:18:14.183 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:14.174+0000 7fcb36549700 1 --2- 172.21.15.170:0/3178593439 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb301224c0 0x7fcb30122ae0 secure :-1 s=THROTTLE_DONE pgs=95 cs=0 l=1 rx=0x7fcb20002f30 tx=0x7fcb20007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:14.183 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:14.174+0000 7fcb35547700 1 -- 172.21.15.170:0/3178593439 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb200113f0 con 0x7fcb301224c0 2020-03-19T21:18:14.183 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:14.174+0000 7fcb36549700 1 --2- 172.21.15.170:0/3178593439 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb301224c0 0x7fcb30122ae0 secure :-1 s=THROTTLE_DONE pgs=95 cs=0 l=1 rx=0x7fcb20002f30 tx=0x7fcb20007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:14.183 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:14.174+0000 7fcb35547700 1 -- 172.21.15.170:0/3178593439 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fcb200116d0 con 0x7fcb301224c0 2020-03-19T21:18:14.183 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:14.174+0000 7fcb387ad700 1 -- 172.21.15.170:0/3178593439 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb301224c0 msgr2=0x7fcb30122ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:14.183 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:14.174+0000 7fcb387ad700 1 --2- 172.21.15.170:0/3178593439 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb301224c0 0x7fcb30122ae0 secure :-1 s=READY pgs=95 cs=0 l=1 rx=0x7fcb20002f30 tx=0x7fcb20007a40).stop 2020-03-19T21:18:14.183 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:14.174+0000 7fcb387ad700 1 -- 172.21.15.170:0/3178593439 shutdown_connections 2020-03-19T21:18:14.184 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:14.174+0000 7fcb387ad700 1 --2- 172.21.15.170:0/3178593439 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb301224c0 0x7fcb30122ae0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:14.184 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:14.174+0000 7fcb387ad700 1 -- 172.21.15.170:0/3178593439 >> 172.21.15.170:0/3178593439 conn(0x7fcb30091390 msgr2=0x7fcb30091770 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:14.184 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:14.174+0000 7fcb387ad700 1 -- 172.21.15.170:0/3178593439 shutdown_connections 2020-03-19T21:18:14.184 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:14.174+0000 7fcb387ad700 1 -- 172.21.15.170:0/3178593439 wait complete. 2020-03-19T21:18:14.184 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:14.175+0000 7fcb387ad700 1 Processor -- start 2020-03-19T21:18:14.184 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:14.175+0000 7fcb387ad700 1 -- start start 2020-03-19T21:18:14.184 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:14.175+0000 7fcb387ad700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fcb301224c0 0x7fcb3011ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:14.185 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:14.175+0000 7fcb387ad700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb3010e620 con 0x7fcb30122f40 2020-03-19T21:18:14.185 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:14.175+0000 7fcb387ad700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb3011f7f0 con 0x7fcb301224c0 2020-03-19T21:18:14.185 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:14.176+0000 7fcb36549700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fcb301224c0 0x7fcb3011ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:14.185 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:14.176+0000 7fcb35d48700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fcb30122f40 0x7fcb3011f0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:53628/0 (socket says 172.21.15.170:53628) 2020-03-19T21:18:14.185 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:14.176+0000 7fcb35d48700 1 -- 172.21.15.170:0/2724155738 learned_addr learned my addr 172.21.15.170:0/2724155738 (peer_addr_for_me v1:172.21.15.170:0/0) 2020-03-19T21:18:14.185 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:14.176+0000 7fcb36549700 1 --2- 172.21.15.170:0/2724155738 >> v2:172.21.15.170:3300/0 conn(0x7fcb301224c0 0x7fcb3011ec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:52866/0 (socket says 172.21.15.170:52866) 2020-03-19T21:18:14.185 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:14.176+0000 7fcb36549700 1 -- 172.21.15.170:0/2724155738 >> v1:172.21.15.170:6789/0 conn(0x7fcb30122f40 legacy=0x7fcb3011f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:14.186 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:14.176+0000 7fcb1f7fe700 1 -- 172.21.15.170:0/2724155738 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4026019676 0 0) 0x7fcb3010e620 con 0x7fcb30122f40 2020-03-19T21:18:14.186 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:14.176+0000 7fcb36549700 1 -- 172.21.15.170:0/2724155738 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb3010f6f0 con 0x7fcb301224c0 2020-03-19T21:18:14.186 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:14.176+0000 7fcb36549700 1 --2- 172.21.15.170:0/2724155738 >> v2:172.21.15.170:3300/0 conn(0x7fcb301224c0 0x7fcb3011ec80 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7fcb3010ebd0 tx=0x7fcb20007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:14.186 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:14.176+0000 7fcb36549700 1 --2- 172.21.15.170:0/2724155738 >> v2:172.21.15.170:3300/0 conn(0x7fcb301224c0 0x7fcb3011ec80 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7fcb3010ebd0 tx=0x7fcb20007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:14.186 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:14.176+0000 7fcb36549700 1 --2- 172.21.15.170:0/2724155738 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb301224c0 0x7fcb3011ec80 secure :-1 s=READY pgs=97 cs=0 l=1 rx=0x7fcb3010ebd0 tx=0x7fcb20007a40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:14.186 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:14.176+0000 7fcb36549700 1 --2- 172.21.15.170:0/2724155738 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb301224c0 0x7fcb3011ec80 secure :-1 s=THROTTLE_DONE pgs=97 cs=0 l=1 rx=0x7fcb3010ebd0 tx=0x7fcb20007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:14.187 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:14.176+0000 7fcb1f7fe700 1 -- 172.21.15.170:0/2724155738 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fcb20011d80 con 0x7fcb301224c0 2020-03-19T21:18:14.187 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:14.176+0000 7fcb36549700 1 --2- 172.21.15.170:0/2724155738 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb301224c0 0x7fcb3011ec80 secure :-1 s=THROTTLE_DONE pgs=97 cs=0 l=1 rx=0x7fcb3010ebd0 tx=0x7fcb20007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:14.187 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:14.176+0000 7fcb36549700 1 --2- 172.21.15.170:0/2724155738 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb301224c0 0x7fcb3011ec80 secure :-1 s=THROTTLE_DONE pgs=97 cs=0 l=1 rx=0x7fcb3010ebd0 tx=0x7fcb20007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:14.187 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:14.176+0000 7fcb1f7fe700 1 -- 172.21.15.170:0/2724155738 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb200056c0 con 0x7fcb301224c0 2020-03-19T21:18:14.187 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:14.176+0000 7fcb387ad700 1 -- 172.21.15.170:0/2724155738 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb3010f990 con 0x7fcb301224c0 2020-03-19T21:18:14.187 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:14.176+0000 7fcb1f7fe700 1 -- 172.21.15.170:0/2724155738 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fcb20005960 con 0x7fcb301224c0 2020-03-19T21:18:14.187 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:14.176+0000 7fcb387ad700 1 -- 172.21.15.170:0/2724155738 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb3010da20 con 0x7fcb301224c0 2020-03-19T21:18:14.188 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:14.176+0000 7fcb36549700 1 --2- 172.21.15.170:0/2724155738 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb301224c0 0x7fcb3011ec80 secure :-1 s=THROTTLE_DONE pgs=97 cs=0 l=1 rx=0x7fcb3010ebd0 tx=0x7fcb20007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:14.188 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:14.177+0000 7fcb36549700 1 --2- 172.21.15.170:0/2724155738 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb301224c0 0x7fcb3011ec80 secure :-1 s=THROTTLE_DONE pgs=97 cs=0 l=1 rx=0x7fcb3010ebd0 tx=0x7fcb20007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:14.188 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:14.177+0000 7fcb1f7fe700 1 -- 172.21.15.170:0/2724155738 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 4) v1 ==== 29603+0+0 (secure 0 0 0) 0x7fcb20005bd0 con 0x7fcb301224c0 2020-03-19T21:18:14.188 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:14.177+0000 7fcb1f7fe700 1 --2- 172.21.15.170:0/2724155738 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7fcb10028280 0x7fcb1002a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:14.188 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:14.177+0000 7fcb1f7fe700 1 -- 172.21.15.170:0/2724155738 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3403+0+0 (secure 0 0 0) 0x7fcb20036ac0 con 0x7fcb301224c0 2020-03-19T21:18:14.188 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:14.177+0000 7fcb35d48700 1 --2- 172.21.15.170:0/2724155738 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7fcb10028280 0x7fcb1002a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:14.188 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:14.177+0000 7fcb35d48700 1 --2- 172.21.15.170:0/2724155738 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7fcb10028280 0x7fcb1002a740 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7fcb2c005500 tx=0x7fcb2c0016e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:14.189 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:14.177+0000 7fcb35d48700 1 --2- 172.21.15.170:0/2724155738 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7fcb10028280 0x7fcb1002a740 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7fcb2c005500 tx=0x7fcb2c0016e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:14.189 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:14.177+0000 7fcb35d48700 1 --2- 172.21.15.170:0/2724155738 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7fcb10028280 0x7fcb1002a740 secure :-1 s=READY pgs=24 cs=0 l=1 rx=0x7fcb2c005500 tx=0x7fcb2c0016e0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:14.189 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:14.177+0000 7fcb387ad700 1 -- 172.21.15.170:0/2724155738 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb30090b60 con 0x7fcb301224c0 2020-03-19T21:18:14.189 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:14.181+0000 7fcb36549700 1 --2- 172.21.15.170:0/2724155738 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb301224c0 0x7fcb3011ec80 secure :-1 s=THROTTLE_DONE pgs=97 cs=0 l=1 rx=0x7fcb3010ebd0 tx=0x7fcb20007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:14.190 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:14.181+0000 7fcb1f7fe700 1 -- 172.21.15.170:0/2724155738 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+126772 (secure 0 0 0) 0x7fcb30090b60 con 0x7fcb301224c0 2020-03-19T21:18:14.370 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:14.368+0000 7fcb387ad700 1 -- 172.21.15.170:0/2724155738 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fcb30090b60 con 0x7fcb301224c0 2020-03-19T21:18:15.178 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.176+0000 7fcb36549700 1 --2- 172.21.15.170:0/2724155738 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb301224c0 0x7fcb3011ec80 secure :-1 s=READY pgs=97 cs=0 l=1 rx=0x7fcb3010ebd0 tx=0x7fcb20007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:15.195 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.193+0000 7fcb36549700 1 --2- 172.21.15.170:0/2724155738 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb301224c0 0x7fcb3011ec80 secure :-1 s=THROTTLE_DONE pgs=97 cs=0 l=1 rx=0x7fcb3010ebd0 tx=0x7fcb20007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:15.195 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.193+0000 7fcb1f7fe700 1 -- 172.21.15.170:0/2724155738 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v3) v1 ==== 118+0+0 (secure 0 0 0) 0x7fcb30090b60 con 0x7fcb301224c0 2020-03-19T21:18:15.198 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.195+0000 7fcb387ad700 1 -- 172.21.15.170:0/2724155738 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7fcb10028280 msgr2=0x7fcb1002a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:15.198 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.195+0000 7fcb387ad700 1 --2- 172.21.15.170:0/2724155738 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7fcb10028280 0x7fcb1002a740 secure :-1 s=READY pgs=24 cs=0 l=1 rx=0x7fcb2c005500 tx=0x7fcb2c0016e0).stop 2020-03-19T21:18:15.198 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.195+0000 7fcb387ad700 1 -- 172.21.15.170:0/2724155738 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb301224c0 msgr2=0x7fcb3011ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:15.198 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.195+0000 7fcb387ad700 1 --2- 172.21.15.170:0/2724155738 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb301224c0 0x7fcb3011ec80 secure :-1 s=READY pgs=97 cs=0 l=1 rx=0x7fcb3010ebd0 tx=0x7fcb20007a40).stop 2020-03-19T21:18:15.199 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.195+0000 7fcb387ad700 1 -- 172.21.15.170:0/2724155738 shutdown_connections 2020-03-19T21:18:15.199 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.195+0000 7fcb387ad700 1 --2- 172.21.15.170:0/2724155738 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7fcb10028280 0x7fcb1002a740 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:15.199 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.195+0000 7fcb387ad700 1 --2- 172.21.15.170:0/2724155738 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb301224c0 0x7fcb3011ec80 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:15.199 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.195+0000 7fcb387ad700 1 -- 172.21.15.170:0/2724155738 >> 172.21.15.170:0/2724155738 conn(0x7fcb30091390 msgr2=0x7fcb30093d40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:15.199 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.195+0000 7fcb387ad700 1 -- 172.21.15.170:0/2724155738 shutdown_connections 2020-03-19T21:18:15.200 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.195+0000 7fcb387ad700 1 -- 172.21.15.170:0/2724155738 wait complete. 2020-03-19T21:18:15.210 INFO:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-03-19T21:18:15.327 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.319+0000 7f4fc9b4b700 1 Processor -- start 2020-03-19T21:18:15.328 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.319+0000 7f4fc9b4b700 1 -- start start 2020-03-19T21:18:15.328 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.319+0000 7f4fc9b4b700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f4fc4065d30 0x7f4fc4066110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T21:18:15.328 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.319+0000 7f4fc9b4b700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fc4090af0 con 0x7f4fc4066570 2020-03-19T21:18:15.328 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.319+0000 7f4fc9b4b700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fc408eec0 con 0x7f4fc4065d30 2020-03-19T21:18:15.328 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.319+0000 7f4fc37fe700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f4fc4065d30 0x7f4fc4066110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:15.329 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.319+0000 7f4fc2ffd700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f4fc4066570 0x7f4fc408d000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:53634/0 (socket says 172.21.15.170:53634) 2020-03-19T21:18:15.329 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.319+0000 7f4fc2ffd700 1 -- 172.21.15.170:0/3966874068 learned_addr learned my addr 172.21.15.170:0/3966874068 (peer_addr_for_me v1:172.21.15.170:0/0) 2020-03-19T21:18:15.329 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.320+0000 7f4fc37fe700 1 -- 172.21.15.170:0/3966874068 >> v1:172.21.15.170:6789/0 conn(0x7f4fc4066570 legacy=0x7f4fc408d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:15.329 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.320+0000 7f4fc27fc700 1 -- 172.21.15.170:0/3966874068 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2325790354 0 0) 0x7f4fc4090af0 con 0x7f4fc4066570 2020-03-19T21:18:15.329 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.320+0000 7f4fc37fe700 1 -- 172.21.15.170:0/3966874068 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fc4060ae0 con 0x7f4fc4065d30 2020-03-19T21:18:15.329 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.320+0000 7f4fc37fe700 1 --2- 172.21.15.170:0/3966874068 >> v2:172.21.15.170:3300/0 conn(0x7f4fc4065d30 0x7f4fc4066110 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f4fb4002f30 tx=0x7f4fb4007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:15.330 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.320+0000 7f4fc37fe700 1 --2- 172.21.15.170:0/3966874068 >> v2:172.21.15.170:3300/0 conn(0x7f4fc4065d30 0x7f4fc4066110 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f4fb4002f30 tx=0x7f4fb4007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:15.330 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.320+0000 7f4fc37fe700 1 --2- 172.21.15.170:0/3966874068 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4fc4065d30 0x7f4fc4066110 secure :-1 s=READY pgs=99 cs=0 l=1 rx=0x7f4fb4002f30 tx=0x7f4fb4007a40).ready entity=mon.0 client_cookie=7477a6f165059605 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:15.330 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.320+0000 7f4fc37fe700 1 --2- 172.21.15.170:0/3966874068 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4fc4065d30 0x7f4fc4066110 secure :-1 s=THROTTLE_DONE pgs=99 cs=0 l=1 rx=0x7f4fb4002f30 tx=0x7f4fb4007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:15.330 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.320+0000 7f4fc27fc700 1 -- 172.21.15.170:0/3966874068 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f4fb4008e80 con 0x7f4fc4065d30 2020-03-19T21:18:15.330 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.320+0000 7f4fc37fe700 1 --2- 172.21.15.170:0/3966874068 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4fc4065d30 0x7f4fc4066110 secure :-1 s=THROTTLE_DONE pgs=99 cs=0 l=1 rx=0x7f4fb4002f30 tx=0x7f4fb4007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:15.330 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.320+0000 7f4fc27fc700 1 -- 172.21.15.170:0/3966874068 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fb40113f0 con 0x7f4fc4065d30 2020-03-19T21:18:15.331 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.320+0000 7f4fc37fe700 1 --2- 172.21.15.170:0/3966874068 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4fc4065d30 0x7f4fc4066110 secure :-1 s=THROTTLE_DONE pgs=99 cs=0 l=1 rx=0x7f4fb4002f30 tx=0x7f4fb4007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:15.331 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.320+0000 7f4fc27fc700 1 -- 172.21.15.170:0/3966874068 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f4fb40116d0 con 0x7f4fc4065d30 2020-03-19T21:18:15.331 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.320+0000 7f4fc9b4b700 1 -- 172.21.15.170:0/3966874068 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4fc4065d30 msgr2=0x7f4fc4066110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:15.331 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.320+0000 7f4fc9b4b700 1 --2- 172.21.15.170:0/3966874068 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4fc4065d30 0x7f4fc4066110 secure :-1 s=READY pgs=99 cs=0 l=1 rx=0x7f4fb4002f30 tx=0x7f4fb4007a40).stop 2020-03-19T21:18:15.331 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.320+0000 7f4fc9b4b700 1 -- 172.21.15.170:0/3966874068 shutdown_connections 2020-03-19T21:18:15.331 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.320+0000 7f4fc9b4b700 1 --2- 172.21.15.170:0/3966874068 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4fc4065d30 0x7f4fc4066110 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:15.331 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.320+0000 7f4fc9b4b700 1 -- 172.21.15.170:0/3966874068 >> 172.21.15.170:0/3966874068 conn(0x7f4fc4091540 msgr2=0x7f4fc4091950 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:15.332 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.320+0000 7f4fc9b4b700 1 -- 172.21.15.170:0/3966874068 shutdown_connections 2020-03-19T21:18:15.332 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.320+0000 7f4fc9b4b700 1 -- 172.21.15.170:0/3966874068 wait complete. 2020-03-19T21:18:15.332 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.321+0000 7f4fc9b4b700 1 Processor -- start 2020-03-19T21:18:15.332 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.321+0000 7f4fc9b4b700 1 -- start start 2020-03-19T21:18:15.332 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.322+0000 7f4fc9b4b700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f4fc4065d30 0x7f4fc4082590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:15.332 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.322+0000 7f4fc9b4b700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fc4085a40 con 0x7f4fc4066570 2020-03-19T21:18:15.332 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.322+0000 7f4fc9b4b700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fc4085d90 con 0x7f4fc4065d30 2020-03-19T21:18:15.333 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.322+0000 7f4fc37fe700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f4fc4065d30 0x7f4fc4082590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:15.333 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.322+0000 7f4fc2ffd700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f4fc4066570 0x7f4fc40829f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:53638/0 (socket says 172.21.15.170:53638) 2020-03-19T21:18:15.333 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.322+0000 7f4fc2ffd700 1 -- 172.21.15.170:0/1984666410 learned_addr learned my addr 172.21.15.170:0/1984666410 (peer_addr_for_me v1:172.21.15.170:0/0) 2020-03-19T21:18:15.333 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.322+0000 7f4fc37fe700 1 -- 172.21.15.170:0/1984666410 >> v1:172.21.15.170:6789/0 conn(0x7f4fc4066570 legacy=0x7f4fc40829f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:15.333 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.322+0000 7f4fc0ff9700 1 -- 172.21.15.170:0/1984666410 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3795403758 0 0) 0x7f4fc4085a40 con 0x7f4fc4066570 2020-03-19T21:18:15.333 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.322+0000 7f4fc37fe700 1 -- 172.21.15.170:0/1984666410 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fc408bb00 con 0x7f4fc4065d30 2020-03-19T21:18:15.333 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.322+0000 7f4fc37fe700 1 --2- 172.21.15.170:0/1984666410 >> v2:172.21.15.170:3300/0 conn(0x7f4fc4065d30 0x7f4fc4082590 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f4fb4002f30 tx=0x7f4fb4007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:15.333 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.322+0000 7f4fc37fe700 1 --2- 172.21.15.170:0/1984666410 >> v2:172.21.15.170:3300/0 conn(0x7f4fc4065d30 0x7f4fc4082590 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f4fb4002f30 tx=0x7f4fb4007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:15.334 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.322+0000 7f4fc37fe700 1 --2- 172.21.15.170:0/1984666410 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4fc4065d30 0x7f4fc4082590 secure :-1 s=READY pgs=101 cs=0 l=1 rx=0x7f4fb4002f30 tx=0x7f4fb4007a40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:15.334 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.322+0000 7f4fc37fe700 1 --2- 172.21.15.170:0/1984666410 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4fc4065d30 0x7f4fc4082590 secure :-1 s=THROTTLE_DONE pgs=101 cs=0 l=1 rx=0x7f4fb4002f30 tx=0x7f4fb4007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:15.334 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.322+0000 7f4fc0ff9700 1 -- 172.21.15.170:0/1984666410 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f4fb4011dc0 con 0x7f4fc4065d30 2020-03-19T21:18:15.334 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.322+0000 7f4fc37fe700 1 --2- 172.21.15.170:0/1984666410 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4fc4065d30 0x7f4fc4082590 secure :-1 s=THROTTLE_DONE pgs=101 cs=0 l=1 rx=0x7f4fb4002f30 tx=0x7f4fb4007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:15.334 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.322+0000 7f4fc37fe700 1 --2- 172.21.15.170:0/1984666410 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4fc4065d30 0x7f4fc4082590 secure :-1 s=THROTTLE_DONE pgs=101 cs=0 l=1 rx=0x7f4fb4002f30 tx=0x7f4fb4007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:15.334 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.322+0000 7f4fc0ff9700 1 -- 172.21.15.170:0/1984666410 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fb4007150 con 0x7f4fc4065d30 2020-03-19T21:18:15.334 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.322+0000 7f4fc9b4b700 1 -- 172.21.15.170:0/1984666410 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fc40844d0 con 0x7f4fc4065d30 2020-03-19T21:18:15.335 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.322+0000 7f4fc0ff9700 1 -- 172.21.15.170:0/1984666410 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f4fb40073f0 con 0x7f4fc4065d30 2020-03-19T21:18:15.335 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.322+0000 7f4fc9b4b700 1 -- 172.21.15.170:0/1984666410 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fc4081cd0 con 0x7f4fc4065d30 2020-03-19T21:18:15.335 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.323+0000 7f4fc37fe700 1 --2- 172.21.15.170:0/1984666410 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4fc4065d30 0x7f4fc4082590 secure :-1 s=THROTTLE_DONE pgs=101 cs=0 l=1 rx=0x7f4fb4002f30 tx=0x7f4fb4007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:15.335 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.323+0000 7f4fc0ff9700 1 -- 172.21.15.170:0/1984666410 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 4) v1 ==== 29603+0+0 (secure 0 0 0) 0x7f4fb40056c0 con 0x7f4fc4065d30 2020-03-19T21:18:15.335 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.323+0000 7f4fc37fe700 1 --2- 172.21.15.170:0/1984666410 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4fc4065d30 0x7f4fc4082590 secure :-1 s=THROTTLE_DONE pgs=101 cs=0 l=1 rx=0x7f4fb4002f30 tx=0x7f4fb4007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:15.335 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.323+0000 7f4fc0ff9700 1 --2- 172.21.15.170:0/1984666410 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f4fa4028250 0x7f4fa402a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:15.335 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.323+0000 7f4fc0ff9700 1 -- 172.21.15.170:0/1984666410 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3403+0+0 (secure 0 0 0) 0x7f4fb4034690 con 0x7f4fc4065d30 2020-03-19T21:18:15.336 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.323+0000 7f4fc2ffd700 1 --2- 172.21.15.170:0/1984666410 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f4fa4028250 0x7f4fa402a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:15.336 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.323+0000 7f4fc2ffd700 1 --2- 172.21.15.170:0/1984666410 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f4fa4028250 0x7f4fa402a710 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f4fb8005500 tx=0x7f4fb80016f0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:15.336 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.323+0000 7f4fc2ffd700 1 --2- 172.21.15.170:0/1984666410 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f4fa4028250 0x7f4fa402a710 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f4fb8005500 tx=0x7f4fb80016f0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:15.336 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.323+0000 7f4fc2ffd700 1 --2- 172.21.15.170:0/1984666410 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f4fa4028250 0x7f4fa402a710 secure :-1 s=READY pgs=25 cs=0 l=1 rx=0x7f4fb8005500 tx=0x7f4fb80016f0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:15.336 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.323+0000 7f4fc9b4b700 1 -- 172.21.15.170:0/1984666410 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fc4056db0 con 0x7f4fc4065d30 2020-03-19T21:18:15.336 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.327+0000 7f4fc37fe700 1 --2- 172.21.15.170:0/1984666410 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4fc4065d30 0x7f4fc4082590 secure :-1 s=THROTTLE_DONE pgs=101 cs=0 l=1 rx=0x7f4fb4002f30 tx=0x7f4fb4007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:15.337 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.327+0000 7f4fc0ff9700 1 -- 172.21.15.170:0/1984666410 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+126772 (secure 0 0 0) 0x7f4fb4009e60 con 0x7f4fc4065d30 2020-03-19T21:18:15.511 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.509+0000 7f4fc9b4b700 1 -- 172.21.15.170:0/1984666410 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4fc406aaf0 con 0x7f4fc4065d30 2020-03-19T21:18:15.512 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.509+0000 7f4fc37fe700 1 --2- 172.21.15.170:0/1984666410 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4fc4065d30 0x7f4fc4082590 secure :-1 s=THROTTLE_DONE pgs=101 cs=0 l=1 rx=0x7f4fb4002f30 tx=0x7f4fb4007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:15.512 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.510+0000 7f4fc0ff9700 1 -- 172.21.15.170:0/1984666410 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+1420 (secure 0 0 0) 0x7f4fb40155a0 con 0x7f4fc4065d30 2020-03-19T21:18:15.512 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:18:15.512 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":3,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":3,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-03-19T21:18:13.514585+0000","modified":"2020-03-19T21:18:15.191871+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[],"up":{},"failed":[],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2020-03-19T21:18:15.513 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.511+0000 7f4fc9b4b700 1 -- 172.21.15.170:0/1984666410 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f4fa4028250 msgr2=0x7f4fa402a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:15.514 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.511+0000 7f4fc9b4b700 1 --2- 172.21.15.170:0/1984666410 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f4fa4028250 0x7f4fa402a710 secure :-1 s=READY pgs=25 cs=0 l=1 rx=0x7f4fb8005500 tx=0x7f4fb80016f0).stop 2020-03-19T21:18:15.514 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.511+0000 7f4fc9b4b700 1 -- 172.21.15.170:0/1984666410 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4fc4065d30 msgr2=0x7f4fc4082590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:15.514 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.511+0000 7f4fc9b4b700 1 --2- 172.21.15.170:0/1984666410 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4fc4065d30 0x7f4fc4082590 secure :-1 s=READY pgs=101 cs=0 l=1 rx=0x7f4fb4002f30 tx=0x7f4fb4007a40).stop 2020-03-19T21:18:15.514 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.511+0000 7f4fc9b4b700 1 -- 172.21.15.170:0/1984666410 shutdown_connections 2020-03-19T21:18:15.514 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.511+0000 7f4fc9b4b700 1 --2- 172.21.15.170:0/1984666410 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f4fa4028250 0x7f4fa402a710 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:15.514 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.511+0000 7f4fc9b4b700 1 --2- 172.21.15.170:0/1984666410 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4fc4065d30 0x7f4fc4082590 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:15.515 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.511+0000 7f4fc9b4b700 1 -- 172.21.15.170:0/1984666410 >> 172.21.15.170:0/1984666410 conn(0x7f4fc4091540 msgr2=0x7f4fc4062e80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:15.515 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.511+0000 7f4fc9b4b700 1 -- 172.21.15.170:0/1984666410 shutdown_connections 2020-03-19T21:18:15.515 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.511+0000 7f4fc9b4b700 1 -- 172.21.15.170:0/1984666410 wait complete. 2020-03-19T21:18:15.515 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 3 2020-03-19T21:18:15.523 INFO:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-03-19T21:18:15.634 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.626+0000 7f85824f5700 1 Processor -- start 2020-03-19T21:18:15.634 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.626+0000 7f85824f5700 1 -- start start 2020-03-19T21:18:15.634 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.626+0000 7f85824f5700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f857c125290 0x7f857c127680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T21:18:15.635 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.626+0000 7f85824f5700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f857c08e840 con 0x7f857c1223d0 2020-03-19T21:18:15.635 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.626+0000 7f85824f5700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f857c091060 con 0x7f857c125290 2020-03-19T21:18:15.635 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.627+0000 7f857b7fe700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f857c125290 0x7f857c127680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:15.635 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.627+0000 7f857bfff700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f857c1223d0 0x7f857c1227b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:53642/0 (socket says 172.21.15.170:53642) 2020-03-19T21:18:15.635 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.627+0000 7f857bfff700 1 -- 172.21.15.170:0/388435427 learned_addr learned my addr 172.21.15.170:0/388435427 (peer_addr_for_me v1:172.21.15.170:0/0) 2020-03-19T21:18:15.635 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.627+0000 7f857b7fe700 1 -- 172.21.15.170:0/388435427 >> v1:172.21.15.170:6789/0 conn(0x7f857c1223d0 legacy=0x7f857c1227b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:15.635 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.627+0000 7f857b7fe700 1 -- 172.21.15.170:0/388435427 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f857c121f30 con 0x7f857c125290 2020-03-19T21:18:15.636 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.627+0000 7f857affd700 1 -- 172.21.15.170:0/388435427 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3977445927 0 0) 0x7f857c08e840 con 0x7f857c1223d0 2020-03-19T21:18:15.636 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.627+0000 7f857b7fe700 1 --2- 172.21.15.170:0/388435427 >> v2:172.21.15.170:3300/0 conn(0x7f857c125290 0x7f857c127680 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f8570002f30 tx=0x7f8570007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:15.636 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.627+0000 7f857b7fe700 1 --2- 172.21.15.170:0/388435427 >> v2:172.21.15.170:3300/0 conn(0x7f857c125290 0x7f857c127680 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f8570002f30 tx=0x7f8570007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:15.636 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.627+0000 7f857b7fe700 1 --2- 172.21.15.170:0/388435427 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f857c125290 0x7f857c127680 secure :-1 s=READY pgs=103 cs=0 l=1 rx=0x7f8570002f30 tx=0x7f8570007a40).ready entity=mon.0 client_cookie=61aab41b15838d3b server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:15.636 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.627+0000 7f857b7fe700 1 --2- 172.21.15.170:0/388435427 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f857c125290 0x7f857c127680 secure :-1 s=THROTTLE_DONE pgs=103 cs=0 l=1 rx=0x7f8570002f30 tx=0x7f8570007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:15.636 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.627+0000 7f857affd700 1 -- 172.21.15.170:0/388435427 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f8570008e80 con 0x7f857c125290 2020-03-19T21:18:15.636 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.627+0000 7f857b7fe700 1 --2- 172.21.15.170:0/388435427 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f857c125290 0x7f857c127680 secure :-1 s=THROTTLE_DONE pgs=103 cs=0 l=1 rx=0x7f8570002f30 tx=0x7f8570007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:15.637 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.627+0000 7f857affd700 1 -- 172.21.15.170:0/388435427 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85700113f0 con 0x7f857c125290 2020-03-19T21:18:15.637 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.627+0000 7f857b7fe700 1 --2- 172.21.15.170:0/388435427 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f857c125290 0x7f857c127680 secure :-1 s=THROTTLE_DONE pgs=103 cs=0 l=1 rx=0x7f8570002f30 tx=0x7f8570007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:15.637 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.627+0000 7f857affd700 1 -- 172.21.15.170:0/388435427 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f85700116d0 con 0x7f857c125290 2020-03-19T21:18:15.637 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.627+0000 7f85824f5700 1 -- 172.21.15.170:0/388435427 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f857c125290 msgr2=0x7f857c127680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:15.637 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.627+0000 7f85824f5700 1 --2- 172.21.15.170:0/388435427 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f857c125290 0x7f857c127680 secure :-1 s=READY pgs=103 cs=0 l=1 rx=0x7f8570002f30 tx=0x7f8570007a40).stop 2020-03-19T21:18:15.637 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.628+0000 7f85824f5700 1 -- 172.21.15.170:0/388435427 shutdown_connections 2020-03-19T21:18:15.637 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.628+0000 7f85824f5700 1 --2- 172.21.15.170:0/388435427 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f857c125290 0x7f857c127680 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:15.638 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.628+0000 7f85824f5700 1 -- 172.21.15.170:0/388435427 >> 172.21.15.170:0/388435427 conn(0x7f857c091540 msgr2=0x7f857c091950 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:15.638 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.628+0000 7f85824f5700 1 -- 172.21.15.170:0/388435427 shutdown_connections 2020-03-19T21:18:15.638 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.628+0000 7f85824f5700 1 -- 172.21.15.170:0/388435427 wait complete. 2020-03-19T21:18:15.638 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.628+0000 7f85824f5700 1 Processor -- start 2020-03-19T21:18:15.638 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.628+0000 7f85824f5700 1 -- start start 2020-03-19T21:18:15.638 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.629+0000 7f85824f5700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f857c1223d0 0x7f857c066640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:15.638 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.629+0000 7f85824f5700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f857c062400 con 0x7f857c125290 2020-03-19T21:18:15.639 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.629+0000 7f85824f5700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f857c0615f0 con 0x7f857c1223d0 2020-03-19T21:18:15.639 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.629+0000 7f857bfff700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f857c1223d0 0x7f857c066640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:15.639 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.629+0000 7f857b7fe700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f857c125290 0x7f857c066aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:53648/0 (socket says 172.21.15.170:53648) 2020-03-19T21:18:15.641 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.629+0000 7f857b7fe700 1 -- 172.21.15.170:0/923057709 learned_addr learned my addr 172.21.15.170:0/923057709 (peer_addr_for_me v1:172.21.15.170:0/0) 2020-03-19T21:18:15.642 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.629+0000 7f857bfff700 1 -- 172.21.15.170:0/923057709 >> v1:172.21.15.170:6789/0 conn(0x7f857c125290 legacy=0x7f857c066aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:15.642 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.629+0000 7f857bfff700 1 -- 172.21.15.170:0/923057709 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f857c060a00 con 0x7f857c1223d0 2020-03-19T21:18:15.642 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.629+0000 7f85797fa700 1 -- 172.21.15.170:0/923057709 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2204152735 0 0) 0x7f857c062400 con 0x7f857c125290 2020-03-19T21:18:15.642 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.629+0000 7f857bfff700 1 --2- 172.21.15.170:0/923057709 >> v2:172.21.15.170:3300/0 conn(0x7f857c1223d0 0x7f857c066640 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f857c11f6a0 tx=0x7f856c010930).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:15.642 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.629+0000 7f857bfff700 1 --2- 172.21.15.170:0/923057709 >> v2:172.21.15.170:3300/0 conn(0x7f857c1223d0 0x7f857c066640 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f857c11f6a0 tx=0x7f856c010930).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:15.642 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.629+0000 7f857bfff700 1 --2- 172.21.15.170:0/923057709 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f857c1223d0 0x7f857c066640 secure :-1 s=READY pgs=105 cs=0 l=1 rx=0x7f857c11f6a0 tx=0x7f856c010930).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:15.643 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.630+0000 7f857bfff700 1 --2- 172.21.15.170:0/923057709 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f857c1223d0 0x7f857c066640 secure :-1 s=THROTTLE_DONE pgs=105 cs=0 l=1 rx=0x7f857c11f6a0 tx=0x7f856c010930).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:15.643 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.630+0000 7f85797fa700 1 -- 172.21.15.170:0/923057709 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f856c005b20 con 0x7f857c1223d0 2020-03-19T21:18:15.643 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.630+0000 7f857bfff700 1 --2- 172.21.15.170:0/923057709 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f857c1223d0 0x7f857c066640 secure :-1 s=THROTTLE_DONE pgs=105 cs=0 l=1 rx=0x7f857c11f6a0 tx=0x7f856c010930).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:15.643 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.630+0000 7f857bfff700 1 --2- 172.21.15.170:0/923057709 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f857c1223d0 0x7f857c066640 secure :-1 s=THROTTLE_DONE pgs=105 cs=0 l=1 rx=0x7f857c11f6a0 tx=0x7f856c010930).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:15.643 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.630+0000 7f85797fa700 1 -- 172.21.15.170:0/923057709 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f856c005c80 con 0x7f857c1223d0 2020-03-19T21:18:15.643 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.630+0000 7f85824f5700 1 -- 172.21.15.170:0/923057709 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f857c061a30 con 0x7f857c1223d0 2020-03-19T21:18:15.643 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.630+0000 7f85797fa700 1 -- 172.21.15.170:0/923057709 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f856c0060c0 con 0x7f857c1223d0 2020-03-19T21:18:15.644 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.630+0000 7f85824f5700 1 -- 172.21.15.170:0/923057709 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f857c0603f0 con 0x7f857c1223d0 2020-03-19T21:18:15.644 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.630+0000 7f857bfff700 1 --2- 172.21.15.170:0/923057709 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f857c1223d0 0x7f857c066640 secure :-1 s=THROTTLE_DONE pgs=105 cs=0 l=1 rx=0x7f857c11f6a0 tx=0x7f856c010930).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:15.644 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.630+0000 7f85797fa700 1 -- 172.21.15.170:0/923057709 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 4) v1 ==== 29603+0+0 (secure 0 0 0) 0x7f856c00a320 con 0x7f857c1223d0 2020-03-19T21:18:15.644 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.630+0000 7f857bfff700 1 --2- 172.21.15.170:0/923057709 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f857c1223d0 0x7f857c066640 secure :-1 s=THROTTLE_DONE pgs=105 cs=0 l=1 rx=0x7f857c11f6a0 tx=0x7f856c010930).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:15.644 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.630+0000 7f85797fa700 1 --2- 172.21.15.170:0/923057709 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f855c028200 0x7f855c02a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:15.644 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.630+0000 7f85797fa700 1 -- 172.21.15.170:0/923057709 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3403+0+0 (secure 0 0 0) 0x7f856c034070 con 0x7f857c1223d0 2020-03-19T21:18:15.644 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.630+0000 7f857b7fe700 1 --2- 172.21.15.170:0/923057709 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f855c028200 0x7f855c02a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:15.645 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.631+0000 7f857b7fe700 1 --2- 172.21.15.170:0/923057709 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f855c028200 0x7f855c02a6c0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f8570002ed0 tx=0x7f8570007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:15.645 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.631+0000 7f85824f5700 1 -- 172.21.15.170:0/923057709 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f857c060870 con 0x7f857c1223d0 2020-03-19T21:18:15.645 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.631+0000 7f857b7fe700 1 --2- 172.21.15.170:0/923057709 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f855c028200 0x7f855c02a6c0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f8570002ed0 tx=0x7f8570007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:15.645 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.631+0000 7f857b7fe700 1 --2- 172.21.15.170:0/923057709 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f855c028200 0x7f855c02a6c0 secure :-1 s=READY pgs=26 cs=0 l=1 rx=0x7f8570002ed0 tx=0x7f8570007a40).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:15.646 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.634+0000 7f857bfff700 1 --2- 172.21.15.170:0/923057709 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f857c1223d0 0x7f857c066640 secure :-1 s=THROTTLE_DONE pgs=105 cs=0 l=1 rx=0x7f857c11f6a0 tx=0x7f856c010930).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:15.646 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.634+0000 7f85797fa700 1 -- 172.21.15.170:0/923057709 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+126772 (secure 0 0 0) 0x7f856c007700 con 0x7f857c1223d0 2020-03-19T21:18:15.776 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.774+0000 7f85824f5700 1 -- 172.21.15.170:0/923057709 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f857c061c50 con 0x7f857c1223d0 2020-03-19T21:18:15.777 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.774+0000 7f857bfff700 1 --2- 172.21.15.170:0/923057709 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f857c1223d0 0x7f857c066640 secure :-1 s=THROTTLE_DONE pgs=105 cs=0 l=1 rx=0x7f857c11f6a0 tx=0x7f856c010930).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:15.777 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.775+0000 7f85797fa700 1 -- 172.21.15.170:0/923057709 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+8837 (secure 0 0 0) 0x7f856c011d90 con 0x7f857c1223d0 2020-03-19T21:18:15.778 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:18:15.778 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":16,"fsid":"12d0a33b-6b40-4398-90b4-6b5df1da388e","created":"2020-03-19T21:17:59.135747+0000","modified":"2020-03-19T21:18:14.515226+0000","last_up_change":"2020-03-19T21:18:05.158515+0000","last_in_change":"2020-03-19T21:18:05.158515+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":3,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2020-03-19T21:18:08.391604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2020-03-19T21:18:11.286536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2020-03-19T21:18:12.597110+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"32ca3517-1077-4102-9aa4-edfcebe8e3d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":20680},{"type":"v1","addr":"172.21.15.170:6801","nonce":20680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":20680},{"type":"v1","addr":"172.21.15.170:6803","nonce":20680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":20680},{"type":"v1","addr":"172.21.15.170:6807","nonce":20680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":20680},{"type":"v1","addr":"172.21.15.170:6805","nonce":20680}]},"public_addr":"172.21.15.170:6801/20680","cluster_addr":"172.21.15.170:6803/20680","heartbeat_back_addr":"172.21.15.170:6807/20680","heartbeat_front_addr":"172.21.15.170:6805/20680","state":["exists","up"]},{"osd":1,"uuid":"470df64b-60bc-4f5b-9b53-64495cd7c5e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":20681},{"type":"v1","addr":"172.21.15.170:6809","nonce":20681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":20681},{"type":"v1","addr":"172.21.15.170:6811","nonce":20681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":20681},{"type":"v1","addr":"172.21.15.170:6815","nonce":20681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":20681},{"type":"v1","addr":"172.21.15.170:6813","nonce":20681}]},"public_addr":"172.21.15.170:6809/20681","cluster_addr":"172.21.15.170:6811/20681","heartbeat_back_addr":"172.21.15.170:6815/20681","heartbeat_front_addr":"172.21.15.170:6813/20681","state":["exists","up"]},{"osd":2,"uuid":"ccfb9d82-b759-4ff4-8ddf-4f1d741bac5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":20682},{"type":"v1","addr":"172.21.15.170:6817","nonce":20682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":20682},{"type":"v1","addr":"172.21.15.170:6819","nonce":20682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":20682},{"type":"v1","addr":"172.21.15.170:6823","nonce":20682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":20682},{"type":"v1","addr":"172.21.15.170:6821","nonce":20682}]},"public_addr":"172.21.15.170:6817/20682","cluster_addr":"172.21.15.170:6819/20682","heartbeat_back_addr":"172.21.15.170:6823/20682","heartbeat_front_addr":"172.21.15.170:6821/20682","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2020-03-19T21:18:03.177293+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2020-03-19T21:18:03.263179+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2020-03-19T21:18:03.183223+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-03-19T21:18:15.780 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.776+0000 7f85824f5700 1 -- 172.21.15.170:0/923057709 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f855c028200 msgr2=0x7f855c02a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:15.780 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.776+0000 7f85824f5700 1 --2- 172.21.15.170:0/923057709 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f855c028200 0x7f855c02a6c0 secure :-1 s=READY pgs=26 cs=0 l=1 rx=0x7f8570002ed0 tx=0x7f8570007a40).stop 2020-03-19T21:18:15.780 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.776+0000 7f85824f5700 1 -- 172.21.15.170:0/923057709 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f857c1223d0 msgr2=0x7f857c066640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:15.780 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.776+0000 7f85824f5700 1 --2- 172.21.15.170:0/923057709 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f857c1223d0 0x7f857c066640 secure :-1 s=READY pgs=105 cs=0 l=1 rx=0x7f857c11f6a0 tx=0x7f856c010930).stop 2020-03-19T21:18:15.781 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.776+0000 7f85824f5700 1 -- 172.21.15.170:0/923057709 shutdown_connections 2020-03-19T21:18:15.781 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.776+0000 7f85824f5700 1 --2- 172.21.15.170:0/923057709 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f855c028200 0x7f855c02a6c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:15.781 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.776+0000 7f85824f5700 1 --2- 172.21.15.170:0/923057709 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f857c1223d0 0x7f857c066640 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:15.781 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.776+0000 7f85824f5700 1 -- 172.21.15.170:0/923057709 >> 172.21.15.170:0/923057709 conn(0x7f857c091540 msgr2=0x7f857c093310 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:15.782 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.776+0000 7f85824f5700 1 -- 172.21.15.170:0/923057709 shutdown_connections 2020-03-19T21:18:15.782 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.776+0000 7f85824f5700 1 -- 172.21.15.170:0/923057709 wait complete. 2020-03-19T21:18:15.789 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2020-03-19T21:18:15.789 INFO:tasks.ceph.mds.a:Restarting daemon 2020-03-19T21:18:15.790 INFO:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2020-03-19T21:18:15.792 INFO:tasks.ceph.mds.a:Started 2020-03-19T21:18:15.793 INFO:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-03-19T21:18:15.794 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2020-03-19T21:18:15.866 INFO:tasks.ceph.mds.a.smithi170.stdout:starting mds.a at 2020-03-19T21:18:15.901 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.899+0000 7f87af28e700 1 Processor -- start 2020-03-19T21:18:15.902 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.899+0000 7f87af28e700 1 -- start start 2020-03-19T21:18:15.911 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.900+0000 7f87af28e700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f87a8096640 0x7f87a8125eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T21:18:15.912 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.900+0000 7f87af28e700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87a808f2f0 con 0x7f87a8126740 2020-03-19T21:18:15.912 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.900+0000 7f87af28e700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87a8090000 con 0x7f87a8096640 2020-03-19T21:18:15.912 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.900+0000 7f87ad02a700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f87a8096640 0x7f87a8125eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:15.912 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.900+0000 7f87ac829700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f87a8126740 0x7f87a808d5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:53662/0 (socket says 172.21.15.170:53662) 2020-03-19T21:18:15.913 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.900+0000 7f87ad02a700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f87a8096640 0x7f87a8125eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:52896/0 (socket says 172.21.15.170:52896) 2020-03-19T21:18:15.913 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.900+0000 7f87ac829700 1 -- 172.21.15.170:0/2982002220 learned_addr learned my addr 172.21.15.170:0/2982002220 (peer_addr_for_me v1:172.21.15.170:0/0) 2020-03-19T21:18:15.913 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.900+0000 7f87ad02a700 1 -- 172.21.15.170:0/2982002220 >> v1:172.21.15.170:6789/0 conn(0x7f87a8126740 legacy=0x7f87a808d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:15.913 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.900+0000 7f87ad02a700 1 -- 172.21.15.170:0/2982002220 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87a8096a20 con 0x7f87a8096640 2020-03-19T21:18:15.914 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.900+0000 7f87ad02a700 1 --2- 172.21.15.170:0/2982002220 >> v2:172.21.15.170:3300/0 conn(0x7f87a8096640 0x7f87a8125eb0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f8798002f30 tx=0x7f8798007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:15.914 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.900+0000 7f87ad02a700 1 --2- 172.21.15.170:0/2982002220 >> v2:172.21.15.170:3300/0 conn(0x7f87a8096640 0x7f87a8125eb0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f8798002f30 tx=0x7f8798007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:15.914 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.900+0000 7f87ad02a700 1 --2- 172.21.15.170:0/2982002220 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87a8096640 0x7f87a8125eb0 secure :-1 s=READY pgs=111 cs=0 l=1 rx=0x7f8798002f30 tx=0x7f8798007a40).ready entity=mon.0 client_cookie=d054c8619b0f36de server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:15.914 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.900+0000 7f87ad02a700 1 --2- 172.21.15.170:0/2982002220 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87a8096640 0x7f87a8125eb0 secure :-1 s=THROTTLE_DONE pgs=111 cs=0 l=1 rx=0x7f8798002f30 tx=0x7f8798007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:15.915 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.900+0000 7f87a7fff700 1 -- 172.21.15.170:0/2982002220 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f8798008e80 con 0x7f87a8096640 2020-03-19T21:18:15.915 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.900+0000 7f87ad02a700 1 --2- 172.21.15.170:0/2982002220 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87a8096640 0x7f87a8125eb0 secure :-1 s=THROTTLE_DONE pgs=111 cs=0 l=1 rx=0x7f8798002f30 tx=0x7f8798007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:15.915 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.900+0000 7f87ad02a700 1 --2- 172.21.15.170:0/2982002220 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87a8096640 0x7f87a8125eb0 secure :-1 s=THROTTLE_DONE pgs=111 cs=0 l=1 rx=0x7f8798002f30 tx=0x7f8798007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:15.915 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.900+0000 7f87a7fff700 1 -- 172.21.15.170:0/2982002220 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87980113f0 con 0x7f87a8096640 2020-03-19T21:18:15.916 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.900+0000 7f87a7fff700 1 -- 172.21.15.170:0/2982002220 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f8798011710 con 0x7f87a8096640 2020-03-19T21:18:15.916 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.900+0000 7f87af28e700 1 -- 172.21.15.170:0/2982002220 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87a8096640 msgr2=0x7f87a8125eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:15.916 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.900+0000 7f87af28e700 1 --2- 172.21.15.170:0/2982002220 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87a8096640 0x7f87a8125eb0 secure :-1 s=READY pgs=111 cs=0 l=1 rx=0x7f8798002f30 tx=0x7f8798007a40).stop 2020-03-19T21:18:15.917 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.901+0000 7f87af28e700 1 -- 172.21.15.170:0/2982002220 shutdown_connections 2020-03-19T21:18:15.917 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.901+0000 7f87af28e700 1 --2- 172.21.15.170:0/2982002220 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87a8096640 0x7f87a8125eb0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:15.917 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.901+0000 7f87af28e700 1 -- 172.21.15.170:0/2982002220 >> 172.21.15.170:0/2982002220 conn(0x7f87a805bf00 msgr2=0x7f87a80990a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:15.917 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.901+0000 7f87af28e700 1 -- 172.21.15.170:0/2982002220 shutdown_connections 2020-03-19T21:18:15.918 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.901+0000 7f87af28e700 1 -- 172.21.15.170:0/2982002220 wait complete. 2020-03-19T21:18:15.918 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.902+0000 7f87af28e700 1 Processor -- start 2020-03-19T21:18:15.918 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.902+0000 7f87af28e700 1 -- start start 2020-03-19T21:18:15.918 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.902+0000 7f87af28e700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f87a8096640 0x7f87a8091f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:15.919 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.902+0000 7f87af28e700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87a8094f80 con 0x7f87a8126740 2020-03-19T21:18:15.919 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.902+0000 7f87af28e700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87a8094c40 con 0x7f87a8096640 2020-03-19T21:18:15.919 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.902+0000 7f87ad02a700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f87a8096640 0x7f87a8091f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:15.922 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.902+0000 7f87ad02a700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f87a8096640 0x7f87a8091f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:52900/0 (socket says 172.21.15.170:52900) 2020-03-19T21:18:15.923 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.902+0000 7f87ac829700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f87a8126740 0x7f87a80923c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:53666/0 (socket says 172.21.15.170:53666) 2020-03-19T21:18:15.923 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.902+0000 7f87ad02a700 1 -- 172.21.15.170:0/1446010564 learned_addr learned my addr 172.21.15.170:0/1446010564 (peer_addr_for_me v2:172.21.15.170:0/0) 2020-03-19T21:18:15.923 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.902+0000 7f87ad02a700 1 -- 172.21.15.170:0/1446010564 >> v1:172.21.15.170:6789/0 conn(0x7f87a8126740 legacy=0x7f87a80923c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:15.923 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.902+0000 7f87a67fc700 1 -- 172.21.15.170:0/1446010564 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4224165954 0 0) 0x7f87a8094f80 con 0x7f87a8126740 2020-03-19T21:18:15.924 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.902+0000 7f87ad02a700 1 -- 172.21.15.170:0/1446010564 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87a80930e0 con 0x7f87a8096640 2020-03-19T21:18:15.924 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.902+0000 7f87ad02a700 1 --2- 172.21.15.170:0/1446010564 >> v2:172.21.15.170:3300/0 conn(0x7f87a8096640 0x7f87a8091f60 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f8798002f30 tx=0x7f8798007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:15.924 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.902+0000 7f87ad02a700 1 --2- 172.21.15.170:0/1446010564 >> v2:172.21.15.170:3300/0 conn(0x7f87a8096640 0x7f87a8091f60 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f8798002f30 tx=0x7f8798007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:15.924 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.903+0000 7f87ad02a700 1 --2- 172.21.15.170:0/1446010564 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87a8096640 0x7f87a8091f60 secure :-1 s=READY pgs=113 cs=0 l=1 rx=0x7f8798002f30 tx=0x7f8798007a40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:15.925 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.903+0000 7f87ad02a700 1 --2- 172.21.15.170:0/1446010564 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87a8096640 0x7f87a8091f60 secure :-1 s=THROTTLE_DONE pgs=113 cs=0 l=1 rx=0x7f8798002f30 tx=0x7f8798007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:15.925 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.903+0000 7f87a67fc700 1 -- 172.21.15.170:0/1446010564 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f879800f830 con 0x7f87a8096640 2020-03-19T21:18:15.925 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.903+0000 7f87ad02a700 1 --2- 172.21.15.170:0/1446010564 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87a8096640 0x7f87a8091f60 secure :-1 s=THROTTLE_DONE pgs=113 cs=0 l=1 rx=0x7f8798002f30 tx=0x7f8798007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:15.925 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.903+0000 7f87ad02a700 1 --2- 172.21.15.170:0/1446010564 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87a8096640 0x7f87a8091f60 secure :-1 s=THROTTLE_DONE pgs=113 cs=0 l=1 rx=0x7f8798002f30 tx=0x7f8798007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:15.926 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.903+0000 7f87a67fc700 1 -- 172.21.15.170:0/1446010564 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8798007150 con 0x7f87a8096640 2020-03-19T21:18:15.926 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.903+0000 7f87af28e700 1 -- 172.21.15.170:0/1446010564 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87a8095210 con 0x7f87a8096640 2020-03-19T21:18:15.926 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.903+0000 7f87a67fc700 1 -- 172.21.15.170:0/1446010564 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f87980073f0 con 0x7f87a8096640 2020-03-19T21:18:15.926 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.903+0000 7f87af28e700 1 -- 172.21.15.170:0/1446010564 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87a8094580 con 0x7f87a8096640 2020-03-19T21:18:15.927 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.903+0000 7f87ad02a700 1 --2- 172.21.15.170:0/1446010564 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87a8096640 0x7f87a8091f60 secure :-1 s=THROTTLE_DONE pgs=113 cs=0 l=1 rx=0x7f8798002f30 tx=0x7f8798007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:15.927 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.903+0000 7f87a67fc700 1 -- 172.21.15.170:0/1446010564 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 4) v1 ==== 29603+0+0 (secure 0 0 0) 0x7f87980056c0 con 0x7f87a8096640 2020-03-19T21:18:15.927 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.903+0000 7f87ad02a700 1 --2- 172.21.15.170:0/1446010564 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87a8096640 0x7f87a8091f60 secure :-1 s=THROTTLE_DONE pgs=113 cs=0 l=1 rx=0x7f8798002f30 tx=0x7f8798007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:15.928 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.903+0000 7f87a67fc700 1 --2- 172.21.15.170:0/1446010564 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f8788028280 0x7f878802a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:15.928 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.903+0000 7f87a67fc700 1 -- 172.21.15.170:0/1446010564 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3403+0+0 (secure 0 0 0) 0x7f8798034700 con 0x7f87a8096640 2020-03-19T21:18:15.928 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.904+0000 7f87ac829700 1 --2- 172.21.15.170:0/1446010564 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f8788028280 0x7f878802a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:15.928 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.904+0000 7f87ac829700 1 --2- 172.21.15.170:0/1446010564 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f8788028280 0x7f878802a740 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f879c000b80 tx=0x7f879c001650).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:15.929 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.904+0000 7f87ac829700 1 --2- 172.21.15.170:0/1446010564 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f8788028280 0x7f878802a740 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f879c000b80 tx=0x7f879c001650).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:15.929 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.904+0000 7f87ac829700 1 --2- 172.21.15.170:0/1446010564 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f8788028280 0x7f878802a740 secure :-1 s=READY pgs=27 cs=0 l=1 rx=0x7f879c000b80 tx=0x7f879c001650).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:15.929 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.904+0000 7f87af28e700 1 -- 172.21.15.170:0/1446010564 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87a8056db0 con 0x7f87a8096640 2020-03-19T21:18:15.930 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.907+0000 7f87ad02a700 1 --2- 172.21.15.170:0/1446010564 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87a8096640 0x7f87a8091f60 secure :-1 s=THROTTLE_DONE pgs=113 cs=0 l=1 rx=0x7f8798002f30 tx=0x7f8798007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:15.930 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:15.907+0000 7f87a67fc700 1 -- 172.21.15.170:0/1446010564 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+126772 (secure 0 0 0) 0x7f8798009870 con 0x7f87a8096640 2020-03-19T21:18:16.047 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.045+0000 7f87af28e700 1 -- 172.21.15.170:0/1446010564 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f87a806aaf0 con 0x7f87a8096640 2020-03-19T21:18:16.048 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.046+0000 7f87ad02a700 1 --2- 172.21.15.170:0/1446010564 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87a8096640 0x7f87a8091f60 secure :-1 s=THROTTLE_DONE pgs=113 cs=0 l=1 rx=0x7f8798002f30 tx=0x7f8798007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:16.048 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.046+0000 7f87a67fc700 1 -- 172.21.15.170:0/1446010564 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+8837 (secure 0 0 0) 0x7f8798009870 con 0x7f87a8096640 2020-03-19T21:18:16.048 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:18:16.048 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":16,"fsid":"12d0a33b-6b40-4398-90b4-6b5df1da388e","created":"2020-03-19T21:17:59.135747+0000","modified":"2020-03-19T21:18:14.515226+0000","last_up_change":"2020-03-19T21:18:05.158515+0000","last_in_change":"2020-03-19T21:18:05.158515+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":3,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2020-03-19T21:18:08.391604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2020-03-19T21:18:11.286536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2020-03-19T21:18:12.597110+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"32ca3517-1077-4102-9aa4-edfcebe8e3d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":20680},{"type":"v1","addr":"172.21.15.170:6801","nonce":20680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":20680},{"type":"v1","addr":"172.21.15.170:6803","nonce":20680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":20680},{"type":"v1","addr":"172.21.15.170:6807","nonce":20680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":20680},{"type":"v1","addr":"172.21.15.170:6805","nonce":20680}]},"public_addr":"172.21.15.170:6801/20680","cluster_addr":"172.21.15.170:6803/20680","heartbeat_back_addr":"172.21.15.170:6807/20680","heartbeat_front_addr":"172.21.15.170:6805/20680","state":["exists","up"]},{"osd":1,"uuid":"470df64b-60bc-4f5b-9b53-64495cd7c5e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":20681},{"type":"v1","addr":"172.21.15.170:6809","nonce":20681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":20681},{"type":"v1","addr":"172.21.15.170:6811","nonce":20681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":20681},{"type":"v1","addr":"172.21.15.170:6815","nonce":20681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":20681},{"type":"v1","addr":"172.21.15.170:6813","nonce":20681}]},"public_addr":"172.21.15.170:6809/20681","cluster_addr":"172.21.15.170:6811/20681","heartbeat_back_addr":"172.21.15.170:6815/20681","heartbeat_front_addr":"172.21.15.170:6813/20681","state":["exists","up"]},{"osd":2,"uuid":"ccfb9d82-b759-4ff4-8ddf-4f1d741bac5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":20682},{"type":"v1","addr":"172.21.15.170:6817","nonce":20682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":20682},{"type":"v1","addr":"172.21.15.170:6819","nonce":20682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":20682},{"type":"v1","addr":"172.21.15.170:6823","nonce":20682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":20682},{"type":"v1","addr":"172.21.15.170:6821","nonce":20682}]},"public_addr":"172.21.15.170:6817/20682","cluster_addr":"172.21.15.170:6819/20682","heartbeat_back_addr":"172.21.15.170:6823/20682","heartbeat_front_addr":"172.21.15.170:6821/20682","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2020-03-19T21:18:03.177293+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2020-03-19T21:18:03.263179+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2020-03-19T21:18:03.183223+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-03-19T21:18:16.049 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.047+0000 7f87af28e700 1 -- 172.21.15.170:0/1446010564 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f8788028280 msgr2=0x7f878802a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:16.050 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.047+0000 7f87af28e700 1 --2- 172.21.15.170:0/1446010564 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f8788028280 0x7f878802a740 secure :-1 s=READY pgs=27 cs=0 l=1 rx=0x7f879c000b80 tx=0x7f879c001650).stop 2020-03-19T21:18:16.050 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.047+0000 7f87af28e700 1 -- 172.21.15.170:0/1446010564 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87a8096640 msgr2=0x7f87a8091f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:16.050 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.047+0000 7f87af28e700 1 --2- 172.21.15.170:0/1446010564 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87a8096640 0x7f87a8091f60 secure :-1 s=READY pgs=113 cs=0 l=1 rx=0x7f8798002f30 tx=0x7f8798007a40).stop 2020-03-19T21:18:16.050 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.047+0000 7f87af28e700 1 -- 172.21.15.170:0/1446010564 shutdown_connections 2020-03-19T21:18:16.050 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.047+0000 7f87af28e700 1 --2- 172.21.15.170:0/1446010564 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f8788028280 0x7f878802a740 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:16.050 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.047+0000 7f87af28e700 1 --2- 172.21.15.170:0/1446010564 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87a8096640 0x7f87a8091f60 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:16.050 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.047+0000 7f87af28e700 1 -- 172.21.15.170:0/1446010564 >> 172.21.15.170:0/1446010564 conn(0x7f87a805bf00 msgr2=0x7f87a8122970 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:16.051 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.047+0000 7f87af28e700 1 -- 172.21.15.170:0/1446010564 shutdown_connections 2020-03-19T21:18:16.051 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.047+0000 7f87af28e700 1 -- 172.21.15.170:0/1446010564 wait complete. 2020-03-19T21:18:16.059 INFO:tasks.ceph.ceph_manager.ceph:[{u'cache_target_full_ratio_micro': 800000, u'fast_read': False, u'stripe_width': 0, u'flags_names': u'hashpspool', u'tier_of': -1, u'hit_set_grade_decay_rate': 0, u'pg_placement_num': 8, u'use_gmt_hitset': True, u'cache_target_dirty_ratio_micro': 400000, u'create_time': u'2020-03-19T21:18:08.391604+0000', u'snap_epoch': 0, u'erasure_code_profile': u'', u'pg_autoscale_mode': u'off', u'snap_seq': 0, u'expected_num_objects': 0, u'size': 2, u'pg_num_pending': 8, u'auid': 0, u'cache_min_flush_age': 0, u'hit_set_period': 0, u'min_read_recency_for_promote': 0, u'target_max_objects': 0, u'pg_placement_num_target': 8, u'pg_num': 8, u'type': 1, u'grade_table': [], u'pool_name': u'rbd', u'cache_min_evict_age': 0, u'snap_mode': u'selfmanaged', u'pg_num_target': 8, u'cache_mode': u'none', u'min_size': 1, u'cache_target_dirty_high_ratio_micro': 600000, u'object_hash': 2, u'last_pg_merge_meta': {u'ready_epoch': 0, u'source_version': u"0'0", u'source_pgid': u'0.0', u'last_epoch_clean': 0, u'target_version': u"0'0", u'last_epoch_started': 0}, u'write_tier': -1, u'last_force_op_resend_preluminous': u'0', u'pool': 1, u'removed_snaps': u'[]', u'crush_rule': 0, u'tiers': [], u'hit_set_params': {u'type': u'none'}, u'last_force_op_resend': u'0', u'pool_snaps': [], u'quota_max_objects': 0, u'last_force_op_resend_prenautilus': u'0', u'application_metadata': {u'rbd': {}}, u'options': {}, u'hit_set_count': 0, u'flags': 1, u'target_max_bytes': 0, u'quota_max_bytes': 0, u'hit_set_search_last_n': 0, u'last_change': u'12', u'min_write_recency_for_promote': 0, u'read_tier': -1}, {u'cache_target_full_ratio_micro': 800000, u'fast_read': False, u'stripe_width': 0, u'flags_names': u'hashpspool', u'tier_of': -1, u'hit_set_grade_decay_rate': 0, u'pg_placement_num': 8, u'use_gmt_hitset': True, u'cache_target_dirty_ratio_micro': 400000, u'create_time': u'2020-03-19T21:18:11.286536+0000', u'snap_epoch': 0, u'erasure_code_profile': u'', u'pg_autoscale_mode': u'off', u'snap_seq': 0, u'expected_num_objects': 0, u'size': 2, u'pg_num_pending': 8, u'auid': 0, u'cache_min_flush_age': 0, u'hit_set_period': 0, u'min_read_recency_for_promote': 0, u'target_max_objects': 0, u'pg_placement_num_target': 8, u'pg_num': 8, u'type': 1, u'grade_table': [], u'pool_name': u'cephfs_metadata', u'cache_min_evict_age': 0, u'snap_mode': u'selfmanaged', u'pg_num_target': 8, u'cache_mode': u'none', u'min_size': 1, u'cache_target_dirty_high_ratio_micro': 600000, u'object_hash': 2, u'last_pg_merge_meta': {u'ready_epoch': 0, u'source_version': u"0'0", u'source_pgid': u'0.0', u'last_epoch_clean': 0, u'target_version': u"0'0", u'last_epoch_started': 0}, u'write_tier': -1, u'last_force_op_resend_preluminous': u'0', u'pool': 2, u'removed_snaps': u'[]', u'crush_rule': 0, u'tiers': [], u'hit_set_params': {u'type': u'none'}, u'last_force_op_resend': u'0', u'pool_snaps': [], u'quota_max_objects': 0, u'last_force_op_resend_prenautilus': u'0', u'application_metadata': {u'cephfs': {u'metadata': u'cephfs'}}, u'options': {u'pg_autoscale_bias': 4, u'recovery_priority': 5, u'pg_num_min': 16}, u'hit_set_count': 0, u'flags': 1, u'target_max_bytes': 0, u'quota_max_bytes': 0, u'hit_set_search_last_n': 0, u'last_change': u'15', u'min_write_recency_for_promote': 0, u'read_tier': -1}, {u'cache_target_full_ratio_micro': 800000, u'fast_read': False, u'stripe_width': 0, u'flags_names': u'hashpspool', u'tier_of': -1, u'hit_set_grade_decay_rate': 0, u'pg_placement_num': 8, u'use_gmt_hitset': True, u'cache_target_dirty_ratio_micro': 400000, u'create_time': u'2020-03-19T21:18:12.597110+0000', u'snap_epoch': 0, u'erasure_code_profile': u'', u'pg_autoscale_mode': u'off', u'snap_seq': 0, u'expected_num_objects': 0, u'size': 2, u'pg_num_pending': 8, u'auid': 0, u'cache_min_flush_age': 0, u'hit_set_period': 0, u'min_read_recency_for_promote': 0, u'target_max_objects': 0, u'pg_placement_num_target': 8, u'pg_num': 8, u'type': 1, u'grade_table': [], u'pool_name': u'cephfs_data', u'cache_min_evict_age': 0, u'snap_mode': u'selfmanaged', u'pg_num_target': 8, u'cache_mode': u'none', u'min_size': 1, u'cache_target_dirty_high_ratio_micro': 600000, u'object_hash': 2, u'last_pg_merge_meta': {u'ready_epoch': 0, u'source_version': u"0'0", u'source_pgid': u'0.0', u'last_epoch_clean': 0, u'target_version': u"0'0", u'last_epoch_started': 0}, u'write_tier': -1, u'last_force_op_resend_preluminous': u'0', u'pool': 3, u'removed_snaps': u'[]', u'crush_rule': 0, u'tiers': [], u'hit_set_params': {u'type': u'none'}, u'last_force_op_resend': u'0', u'pool_snaps': [], u'quota_max_objects': 0, u'last_force_op_resend_prenautilus': u'0', u'application_metadata': {u'cephfs': {u'data': u'cephfs'}}, u'options': {}, u'hit_set_count': 0, u'flags': 1, u'target_max_bytes': 0, u'quota_max_bytes': 0, u'hit_set_search_last_n': 0, u'last_change': u'16', u'min_write_recency_for_promote': 0, u'read_tier': -1}] 2020-03-19T21:18:16.060 INFO:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-03-19T21:18:16.176 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.165+0000 7fa536573700 1 Processor -- start 2020-03-19T21:18:16.176 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.165+0000 7fa536573700 1 -- start start 2020-03-19T21:18:16.176 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.165+0000 7fa536573700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fa530091e80 0x7fa530120520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T21:18:16.176 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.165+0000 7fa536573700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa530091020 con 0x7fa530120980 2020-03-19T21:18:16.176 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.165+0000 7fa536573700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa530120020 con 0x7fa530091e80 2020-03-19T21:18:16.177 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.165+0000 7fa52ffff700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fa530091e80 0x7fa530120520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:16.177 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.165+0000 7fa52ffff700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fa530091e80 0x7fa530120520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:52906/0 (socket says 172.21.15.170:52906) 2020-03-19T21:18:16.177 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.165+0000 7fa52f7fe700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fa530120980 0x7fa53008f290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:53670/0 (socket says 172.21.15.170:53670) 2020-03-19T21:18:16.177 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.165+0000 7fa52ffff700 1 -- 172.21.15.170:0/646761729 learned_addr learned my addr 172.21.15.170:0/646761729 (peer_addr_for_me v2:172.21.15.170:0/0) 2020-03-19T21:18:16.177 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.166+0000 7fa52ffff700 1 -- 172.21.15.170:0/646761729 >> v1:172.21.15.170:6789/0 conn(0x7fa530120980 legacy=0x7fa53008f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:16.178 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.166+0000 7fa52ffff700 1 -- 172.21.15.170:0/646761729 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa530120210 con 0x7fa530091e80 2020-03-19T21:18:16.178 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.166+0000 7fa52ffff700 1 --2- 172.21.15.170:0/646761729 >> v2:172.21.15.170:3300/0 conn(0x7fa530091e80 0x7fa530120520 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7fa520002f30 tx=0x7fa520007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:16.178 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.166+0000 7fa52ffff700 1 --2- 172.21.15.170:0/646761729 >> v2:172.21.15.170:3300/0 conn(0x7fa530091e80 0x7fa530120520 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7fa520002f30 tx=0x7fa520007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:16.178 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.166+0000 7fa52ffff700 1 --2- 172.21.15.170:0/646761729 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa530091e80 0x7fa530120520 secure :-1 s=READY pgs=115 cs=0 l=1 rx=0x7fa520002f30 tx=0x7fa520007a40).ready entity=mon.0 client_cookie=b06dc0daa96c50d2 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:16.178 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.166+0000 7fa52ffff700 1 --2- 172.21.15.170:0/646761729 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa530091e80 0x7fa530120520 secure :-1 s=THROTTLE_DONE pgs=115 cs=0 l=1 rx=0x7fa520002f30 tx=0x7fa520007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:16.178 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.166+0000 7fa52effd700 1 -- 172.21.15.170:0/646761729 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fa5200114b0 con 0x7fa530091e80 2020-03-19T21:18:16.179 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.166+0000 7fa52ffff700 1 --2- 172.21.15.170:0/646761729 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa530091e80 0x7fa530120520 secure :-1 s=THROTTLE_DONE pgs=115 cs=0 l=1 rx=0x7fa520002f30 tx=0x7fa520007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:16.179 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.166+0000 7fa52effd700 1 -- 172.21.15.170:0/646761729 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa520011610 con 0x7fa530091e80 2020-03-19T21:18:16.179 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.166+0000 7fa52ffff700 1 --2- 172.21.15.170:0/646761729 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa530091e80 0x7fa530120520 secure :-1 s=THROTTLE_DONE pgs=115 cs=0 l=1 rx=0x7fa520002f30 tx=0x7fa520007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:16.179 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.166+0000 7fa52effd700 1 -- 172.21.15.170:0/646761729 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fa520011970 con 0x7fa530091e80 2020-03-19T21:18:16.180 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.166+0000 7fa536573700 1 -- 172.21.15.170:0/646761729 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa530091e80 msgr2=0x7fa530120520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:16.180 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.166+0000 7fa536573700 1 --2- 172.21.15.170:0/646761729 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa530091e80 0x7fa530120520 secure :-1 s=READY pgs=115 cs=0 l=1 rx=0x7fa520002f30 tx=0x7fa520007a40).stop 2020-03-19T21:18:16.180 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.166+0000 7fa536573700 1 -- 172.21.15.170:0/646761729 shutdown_connections 2020-03-19T21:18:16.180 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.166+0000 7fa536573700 1 --2- 172.21.15.170:0/646761729 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa530091e80 0x7fa530120520 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:16.180 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.166+0000 7fa536573700 1 -- 172.21.15.170:0/646761729 >> 172.21.15.170:0/646761729 conn(0x7fa530091390 msgr2=0x7fa530091770 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:16.181 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.166+0000 7fa536573700 1 -- 172.21.15.170:0/646761729 shutdown_connections 2020-03-19T21:18:16.181 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.167+0000 7fa536573700 1 -- 172.21.15.170:0/646761729 wait complete. 2020-03-19T21:18:16.181 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.167+0000 7fa536573700 1 Processor -- start 2020-03-19T21:18:16.181 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.167+0000 7fa536573700 1 -- start start 2020-03-19T21:18:16.181 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.168+0000 7fa536573700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fa530091e80 0x7fa53010aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:16.182 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.168+0000 7fa536573700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa53011fc30 con 0x7fa530120980 2020-03-19T21:18:16.182 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.168+0000 7fa536573700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa53011f380 con 0x7fa530091e80 2020-03-19T21:18:16.182 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.168+0000 7fa52ffff700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fa530091e80 0x7fa53010aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:16.182 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.168+0000 7fa52f7fe700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fa530120980 0x7fa53010b330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:53676/0 (socket says 172.21.15.170:53676) 2020-03-19T21:18:16.183 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.168+0000 7fa52f7fe700 1 -- 172.21.15.170:0/719889168 learned_addr learned my addr 172.21.15.170:0/719889168 (peer_addr_for_me v1:172.21.15.170:0/0) 2020-03-19T21:18:16.183 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.168+0000 7fa52ffff700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fa530091e80 0x7fa53010aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:52910/0 (socket says 172.21.15.170:52910) 2020-03-19T21:18:16.183 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.168+0000 7fa52ffff700 1 -- 172.21.15.170:0/719889168 >> v1:172.21.15.170:6789/0 conn(0x7fa530120980 legacy=0x7fa53010b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:16.183 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.168+0000 7fa52d7fa700 1 -- 172.21.15.170:0/719889168 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1962932660 0 0) 0x7fa53011fc30 con 0x7fa530120980 2020-03-19T21:18:16.183 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.168+0000 7fa52ffff700 1 -- 172.21.15.170:0/719889168 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa53011fe60 con 0x7fa530091e80 2020-03-19T21:18:16.184 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.168+0000 7fa52ffff700 1 --2- 172.21.15.170:0/719889168 >> v2:172.21.15.170:3300/0 conn(0x7fa530091e80 0x7fa53010aed0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7fa520002f30 tx=0x7fa520007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:16.184 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.168+0000 7fa52ffff700 1 --2- 172.21.15.170:0/719889168 >> v2:172.21.15.170:3300/0 conn(0x7fa530091e80 0x7fa53010aed0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7fa520002f30 tx=0x7fa520007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:16.185 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.168+0000 7fa52ffff700 1 --2- 172.21.15.170:0/719889168 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa530091e80 0x7fa53010aed0 secure :-1 s=READY pgs=117 cs=0 l=1 rx=0x7fa520002f30 tx=0x7fa520007a40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:16.185 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.168+0000 7fa52ffff700 1 --2- 172.21.15.170:0/719889168 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa530091e80 0x7fa53010aed0 secure :-1 s=THROTTLE_DONE pgs=117 cs=0 l=1 rx=0x7fa520002f30 tx=0x7fa520007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:16.185 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.168+0000 7fa52d7fa700 1 -- 172.21.15.170:0/719889168 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fa52000f780 con 0x7fa530091e80 2020-03-19T21:18:16.186 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.168+0000 7fa52ffff700 1 --2- 172.21.15.170:0/719889168 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa530091e80 0x7fa53010aed0 secure :-1 s=THROTTLE_DONE pgs=117 cs=0 l=1 rx=0x7fa520002f30 tx=0x7fa520007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:16.186 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.168+0000 7fa52ffff700 1 --2- 172.21.15.170:0/719889168 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa530091e80 0x7fa53010aed0 secure :-1 s=THROTTLE_DONE pgs=117 cs=0 l=1 rx=0x7fa520002f30 tx=0x7fa520007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:16.186 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.168+0000 7fa52d7fa700 1 -- 172.21.15.170:0/719889168 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa52000f8e0 con 0x7fa530091e80 2020-03-19T21:18:16.187 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.168+0000 7fa536573700 1 -- 172.21.15.170:0/719889168 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa53010c7a0 con 0x7fa530091e80 2020-03-19T21:18:16.187 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.168+0000 7fa52d7fa700 1 -- 172.21.15.170:0/719889168 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fa520007280 con 0x7fa530091e80 2020-03-19T21:18:16.187 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.168+0000 7fa536573700 1 -- 172.21.15.170:0/719889168 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa530065df0 con 0x7fa530091e80 2020-03-19T21:18:16.187 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.169+0000 7fa52ffff700 1 --2- 172.21.15.170:0/719889168 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa530091e80 0x7fa53010aed0 secure :-1 s=THROTTLE_DONE pgs=117 cs=0 l=1 rx=0x7fa520002f30 tx=0x7fa520007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:16.188 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.169+0000 7fa52d7fa700 1 -- 172.21.15.170:0/719889168 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 4) v1 ==== 29603+0+0 (secure 0 0 0) 0x7fa5200056c0 con 0x7fa530091e80 2020-03-19T21:18:16.188 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.169+0000 7fa52ffff700 1 --2- 172.21.15.170:0/719889168 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa530091e80 0x7fa53010aed0 secure :-1 s=THROTTLE_DONE pgs=117 cs=0 l=1 rx=0x7fa520002f30 tx=0x7fa520007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:16.188 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.169+0000 7fa52d7fa700 1 --2- 172.21.15.170:0/719889168 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7fa510028210 0x7fa51002a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:16.188 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.169+0000 7fa52d7fa700 1 -- 172.21.15.170:0/719889168 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3403+0+0 (secure 0 0 0) 0x7fa520034850 con 0x7fa530091e80 2020-03-19T21:18:16.188 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.169+0000 7fa52f7fe700 1 --2- 172.21.15.170:0/719889168 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7fa510028210 0x7fa51002a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:16.188 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.169+0000 7fa52f7fe700 1 --2- 172.21.15.170:0/719889168 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7fa510028210 0x7fa51002a6d0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7fa524000ef0 tx=0x7fa524001740).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:16.189 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.169+0000 7fa536573700 1 -- 172.21.15.170:0/719889168 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa53010d0f0 con 0x7fa530091e80 2020-03-19T21:18:16.189 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.169+0000 7fa52f7fe700 1 --2- 172.21.15.170:0/719889168 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7fa510028210 0x7fa51002a6d0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7fa524000ef0 tx=0x7fa524001740).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:16.189 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.169+0000 7fa52f7fe700 1 --2- 172.21.15.170:0/719889168 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7fa510028210 0x7fa51002a6d0 secure :-1 s=READY pgs=28 cs=0 l=1 rx=0x7fa524000ef0 tx=0x7fa524001740).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:16.189 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.173+0000 7fa52ffff700 1 --2- 172.21.15.170:0/719889168 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa530091e80 0x7fa53010aed0 secure :-1 s=THROTTLE_DONE pgs=117 cs=0 l=1 rx=0x7fa520002f30 tx=0x7fa520007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:16.189 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.173+0000 7fa52d7fa700 1 -- 172.21.15.170:0/719889168 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+126772 (secure 0 0 0) 0x7fa520007500 con 0x7fa530091e80 2020-03-19T21:18:16.327 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.325+0000 7fa536573700 1 -- 172.21.15.170:0/719889168 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fa530066850 con 0x7fa530091e80 2020-03-19T21:18:16.327 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.325+0000 7fa52ffff700 1 --2- 172.21.15.170:0/719889168 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa530091e80 0x7fa53010aed0 secure :-1 s=THROTTLE_DONE pgs=117 cs=0 l=1 rx=0x7fa520002f30 tx=0x7fa520007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:16.327 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.325+0000 7fa52d7fa700 1 -- 172.21.15.170:0/719889168 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v16) v1 ==== 92+0+10 (secure 0 0 0) 0x7fa520009de0 con 0x7fa530091e80 2020-03-19T21:18:16.328 INFO:teuthology.orchestra.run.smithi170.stdout:pg_num: 8 2020-03-19T21:18:16.329 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.326+0000 7fa536573700 1 -- 172.21.15.170:0/719889168 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7fa510028210 msgr2=0x7fa51002a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:16.329 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.326+0000 7fa536573700 1 --2- 172.21.15.170:0/719889168 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7fa510028210 0x7fa51002a6d0 secure :-1 s=READY pgs=28 cs=0 l=1 rx=0x7fa524000ef0 tx=0x7fa524001740).stop 2020-03-19T21:18:16.329 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.326+0000 7fa536573700 1 -- 172.21.15.170:0/719889168 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa530091e80 msgr2=0x7fa53010aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:16.329 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.326+0000 7fa536573700 1 --2- 172.21.15.170:0/719889168 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa530091e80 0x7fa53010aed0 secure :-1 s=READY pgs=117 cs=0 l=1 rx=0x7fa520002f30 tx=0x7fa520007a40).stop 2020-03-19T21:18:16.330 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.326+0000 7fa536573700 1 -- 172.21.15.170:0/719889168 shutdown_connections 2020-03-19T21:18:16.330 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.326+0000 7fa536573700 1 --2- 172.21.15.170:0/719889168 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7fa510028210 0x7fa51002a6d0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:16.330 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.326+0000 7fa536573700 1 --2- 172.21.15.170:0/719889168 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa530091e80 0x7fa53010aed0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:16.330 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.326+0000 7fa536573700 1 -- 172.21.15.170:0/719889168 >> 172.21.15.170:0/719889168 conn(0x7fa530091390 msgr2=0x7fa53008f290 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:16.330 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.326+0000 7fa536573700 1 -- 172.21.15.170:0/719889168 shutdown_connections 2020-03-19T21:18:16.330 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.326+0000 7fa536573700 1 -- 172.21.15.170:0/719889168 wait complete. 2020-03-19T21:18:16.339 INFO:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2020-03-19T21:18:16.446 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.444+0000 7f8d7c1df700 1 Processor -- start 2020-03-19T21:18:16.446 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.444+0000 7f8d7c1df700 1 -- start start 2020-03-19T21:18:16.451 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.445+0000 7f8d7c1df700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f8d74125840 0x7f8d74127c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T21:18:16.451 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.445+0000 7f8d7c1df700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d741228a0 con 0x7f8d741224c0 2020-03-19T21:18:16.452 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.445+0000 7f8d7c1df700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d74092200 con 0x7f8d74125840 2020-03-19T21:18:16.452 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.445+0000 7f8d7977a700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f8d74125840 0x7f8d74127c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:16.452 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.445+0000 7f8d7977a700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f8d74125840 0x7f8d74127c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:52926/0 (socket says 172.21.15.170:52926) 2020-03-19T21:18:16.452 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.445+0000 7f8d7977a700 1 -- 172.21.15.170:0/627414022 learned_addr learned my addr 172.21.15.170:0/627414022 (peer_addr_for_me v2:172.21.15.170:0/0) 2020-03-19T21:18:16.452 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.445+0000 7f8d79f7b700 1 --1- 172.21.15.170:0/627414022 >> v1:172.21.15.170:6789/0 conn(0x7f8d741224c0 0x7f8d74122ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:53688/0 (socket says 172.21.15.170:53688) 2020-03-19T21:18:16.452 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.445+0000 7f8d7977a700 1 -- 172.21.15.170:0/627414022 >> v1:172.21.15.170:6789/0 conn(0x7f8d741224c0 legacy=0x7f8d74122ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:16.452 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.445+0000 7f8d7977a700 1 -- 172.21.15.170:0/627414022 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d74090b60 con 0x7f8d74125840 2020-03-19T21:18:16.453 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.445+0000 7f8d78f79700 1 -- 172.21.15.170:0/627414022 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3725221743 0 0) 0x7f8d640010d0 con 0x7f8d741224c0 2020-03-19T21:18:16.453 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.445+0000 7f8d7977a700 1 --2- 172.21.15.170:0/627414022 >> v2:172.21.15.170:3300/0 conn(0x7f8d74125840 0x7f8d74127c30 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f8d70002f30 tx=0x7f8d70007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:16.453 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.445+0000 7f8d7977a700 1 --2- 172.21.15.170:0/627414022 >> v2:172.21.15.170:3300/0 conn(0x7f8d74125840 0x7f8d74127c30 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f8d70002f30 tx=0x7f8d70007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:16.453 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.445+0000 7f8d7977a700 1 --2- 172.21.15.170:0/627414022 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8d74125840 0x7f8d74127c30 secure :-1 s=READY pgs=119 cs=0 l=1 rx=0x7f8d70002f30 tx=0x7f8d70007a40).ready entity=mon.0 client_cookie=56b75c036517c30f server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:16.453 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.445+0000 7f8d7977a700 1 --2- 172.21.15.170:0/627414022 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8d74125840 0x7f8d74127c30 secure :-1 s=THROTTLE_DONE pgs=119 cs=0 l=1 rx=0x7f8d70002f30 tx=0x7f8d70007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:16.453 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.445+0000 7f8d78f79700 1 -- 172.21.15.170:0/627414022 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f8d700114b0 con 0x7f8d74125840 2020-03-19T21:18:16.453 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.445+0000 7f8d7977a700 1 --2- 172.21.15.170:0/627414022 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8d74125840 0x7f8d74127c30 secure :-1 s=THROTTLE_DONE pgs=119 cs=0 l=1 rx=0x7f8d70002f30 tx=0x7f8d70007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:16.454 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.445+0000 7f8d78f79700 1 -- 172.21.15.170:0/627414022 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d70011610 con 0x7f8d74125840 2020-03-19T21:18:16.454 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.445+0000 7f8d7977a700 1 --2- 172.21.15.170:0/627414022 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8d74125840 0x7f8d74127c30 secure :-1 s=THROTTLE_DONE pgs=119 cs=0 l=1 rx=0x7f8d70002f30 tx=0x7f8d70007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:16.454 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.445+0000 7f8d7c1df700 1 -- 172.21.15.170:0/627414022 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8d74125840 msgr2=0x7f8d74127c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:16.454 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.445+0000 7f8d78f79700 1 -- 172.21.15.170:0/627414022 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f8d70011930 con 0x7f8d74125840 2020-03-19T21:18:16.454 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.445+0000 7f8d7c1df700 1 --2- 172.21.15.170:0/627414022 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8d74125840 0x7f8d74127c30 secure :-1 s=READY pgs=119 cs=0 l=1 rx=0x7f8d70002f30 tx=0x7f8d70007a40).stop 2020-03-19T21:18:16.454 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.446+0000 7f8d7c1df700 1 -- 172.21.15.170:0/627414022 shutdown_connections 2020-03-19T21:18:16.454 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.446+0000 7f8d7c1df700 1 --2- 172.21.15.170:0/627414022 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8d74125840 0x7f8d74127c30 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:16.455 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.446+0000 7f8d7c1df700 1 -- 172.21.15.170:0/627414022 >> 172.21.15.170:0/627414022 conn(0x7f8d74091390 msgr2=0x7f8d74091770 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:16.455 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.446+0000 7f8d7c1df700 1 -- 172.21.15.170:0/627414022 shutdown_connections 2020-03-19T21:18:16.455 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.446+0000 7f8d7c1df700 1 -- 172.21.15.170:0/627414022 wait complete. 2020-03-19T21:18:16.455 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.447+0000 7f8d7c1df700 1 Processor -- start 2020-03-19T21:18:16.455 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.447+0000 7f8d7c1df700 1 -- start start 2020-03-19T21:18:16.455 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.447+0000 7f8d7c1df700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f8d74125840 0x7f8d7411e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:16.455 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.447+0000 7f8d7c1df700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d74065360 con 0x7f8d741224c0 2020-03-19T21:18:16.456 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.447+0000 7f8d7c1df700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d740650f0 con 0x7f8d74125840 2020-03-19T21:18:16.456 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.447+0000 7f8d79f7b700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f8d741224c0 0x7f8d7411d960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:53692/0 (socket says 172.21.15.170:53692) 2020-03-19T21:18:16.456 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.447+0000 7f8d7977a700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f8d74125840 0x7f8d7411e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:16.456 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.447+0000 7f8d79f7b700 1 -- 172.21.15.170:0/3574523377 learned_addr learned my addr 172.21.15.170:0/3574523377 (peer_addr_for_me v1:172.21.15.170:0/0) 2020-03-19T21:18:16.456 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.447+0000 7f8d7977a700 1 -- 172.21.15.170:0/3574523377 >> v1:172.21.15.170:6789/0 conn(0x7f8d741224c0 legacy=0x7f8d7411d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:16.456 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.447+0000 7f8d62ffd700 1 -- 172.21.15.170:0/3574523377 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 245638392 0 0) 0x7f8d74065360 con 0x7f8d741224c0 2020-03-19T21:18:16.456 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.447+0000 7f8d7977a700 1 -- 172.21.15.170:0/3574523377 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d7410f6c0 con 0x7f8d74125840 2020-03-19T21:18:16.457 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.447+0000 7f8d7977a700 1 --2- 172.21.15.170:0/3574523377 >> v2:172.21.15.170:3300/0 conn(0x7f8d74125840 0x7f8d7411e070 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f8d74065c90 tx=0x7f8d70007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:16.457 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.447+0000 7f8d7977a700 1 --2- 172.21.15.170:0/3574523377 >> v2:172.21.15.170:3300/0 conn(0x7f8d74125840 0x7f8d7411e070 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f8d74065c90 tx=0x7f8d70007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:16.457 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.447+0000 7f8d7977a700 1 --2- 172.21.15.170:0/3574523377 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8d74125840 0x7f8d7411e070 secure :-1 s=READY pgs=121 cs=0 l=1 rx=0x7f8d74065c90 tx=0x7f8d70007a40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:16.457 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.448+0000 7f8d7977a700 1 --2- 172.21.15.170:0/3574523377 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8d74125840 0x7f8d7411e070 secure :-1 s=THROTTLE_DONE pgs=121 cs=0 l=1 rx=0x7f8d74065c90 tx=0x7f8d70007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:16.457 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.448+0000 7f8d62ffd700 1 -- 172.21.15.170:0/3574523377 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f8d70011990 con 0x7f8d74125840 2020-03-19T21:18:16.457 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.448+0000 7f8d7977a700 1 --2- 172.21.15.170:0/3574523377 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8d74125840 0x7f8d7411e070 secure :-1 s=THROTTLE_DONE pgs=121 cs=0 l=1 rx=0x7f8d74065c90 tx=0x7f8d70007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:16.458 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.448+0000 7f8d7977a700 1 --2- 172.21.15.170:0/3574523377 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8d74125840 0x7f8d7411e070 secure :-1 s=THROTTLE_DONE pgs=121 cs=0 l=1 rx=0x7f8d74065c90 tx=0x7f8d70007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:16.458 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.448+0000 7f8d62ffd700 1 -- 172.21.15.170:0/3574523377 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d70007150 con 0x7f8d74125840 2020-03-19T21:18:16.458 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.448+0000 7f8d7c1df700 1 -- 172.21.15.170:0/3574523377 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d7410f2e0 con 0x7f8d74125840 2020-03-19T21:18:16.458 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.448+0000 7f8d62ffd700 1 -- 172.21.15.170:0/3574523377 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f8d700073f0 con 0x7f8d74125840 2020-03-19T21:18:16.458 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.448+0000 7f8d7c1df700 1 -- 172.21.15.170:0/3574523377 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d74065e50 con 0x7f8d74125840 2020-03-19T21:18:16.458 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.448+0000 7f8d7977a700 1 --2- 172.21.15.170:0/3574523377 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8d74125840 0x7f8d7411e070 secure :-1 s=THROTTLE_DONE pgs=121 cs=0 l=1 rx=0x7f8d74065c90 tx=0x7f8d70007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:16.459 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.448+0000 7f8d62ffd700 1 -- 172.21.15.170:0/3574523377 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 4) v1 ==== 29603+0+0 (secure 0 0 0) 0x7f8d700056c0 con 0x7f8d74125840 2020-03-19T21:18:16.459 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.448+0000 7f8d7977a700 1 --2- 172.21.15.170:0/3574523377 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8d74125840 0x7f8d7411e070 secure :-1 s=THROTTLE_DONE pgs=121 cs=0 l=1 rx=0x7f8d74065c90 tx=0x7f8d70007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:16.459 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.448+0000 7f8d62ffd700 1 --2- 172.21.15.170:0/3574523377 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f8d54028280 0x7f8d5402a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:16.459 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.448+0000 7f8d62ffd700 1 -- 172.21.15.170:0/3574523377 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3403+0+0 (secure 0 0 0) 0x7f8d70034680 con 0x7f8d74125840 2020-03-19T21:18:16.459 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.448+0000 7f8d79f7b700 1 --2- 172.21.15.170:0/3574523377 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f8d54028280 0x7f8d5402a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:16.461 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.449+0000 7f8d79f7b700 1 --2- 172.21.15.170:0/3574523377 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f8d54028280 0x7f8d5402a740 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f8d64001180 tx=0x7f8d640017c0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:16.461 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.449+0000 7f8d7c1df700 1 -- 172.21.15.170:0/3574523377 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d74066be0 con 0x7f8d74125840 2020-03-19T21:18:16.461 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.449+0000 7f8d79f7b700 1 --2- 172.21.15.170:0/3574523377 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f8d54028280 0x7f8d5402a740 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f8d64001180 tx=0x7f8d640017c0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:16.461 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.449+0000 7f8d79f7b700 1 --2- 172.21.15.170:0/3574523377 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f8d54028280 0x7f8d5402a740 secure :-1 s=READY pgs=30 cs=0 l=1 rx=0x7f8d64001180 tx=0x7f8d640017c0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:16.462 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.452+0000 7f8d7977a700 1 --2- 172.21.15.170:0/3574523377 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8d74125840 0x7f8d7411e070 secure :-1 s=THROTTLE_DONE pgs=121 cs=0 l=1 rx=0x7f8d74065c90 tx=0x7f8d70007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:16.462 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.452+0000 7f8d62ffd700 1 -- 172.21.15.170:0/3574523377 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+126772 (secure 0 0 0) 0x7f8d700078a0 con 0x7f8d74125840 2020-03-19T21:18:16.592 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.590+0000 7f8d7c1df700 1 -- 172.21.15.170:0/3574523377 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8d74090b60 con 0x7f8d74125840 2020-03-19T21:18:16.592 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.590+0000 7f8d7977a700 1 --2- 172.21.15.170:0/3574523377 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8d74125840 0x7f8d7411e070 secure :-1 s=THROTTLE_DONE pgs=121 cs=0 l=1 rx=0x7f8d74065c90 tx=0x7f8d70007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:16.593 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.590+0000 7f8d62ffd700 1 -- 172.21.15.170:0/3574523377 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v16) v1 ==== 104+0+10 (secure 0 0 0) 0x7f8d74090b60 con 0x7f8d74125840 2020-03-19T21:18:16.593 INFO:teuthology.orchestra.run.smithi170.stdout:pg_num: 8 2020-03-19T21:18:16.594 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.592+0000 7f8d7c1df700 1 -- 172.21.15.170:0/3574523377 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f8d54028280 msgr2=0x7f8d5402a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:16.594 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.592+0000 7f8d7c1df700 1 --2- 172.21.15.170:0/3574523377 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f8d54028280 0x7f8d5402a740 secure :-1 s=READY pgs=30 cs=0 l=1 rx=0x7f8d64001180 tx=0x7f8d640017c0).stop 2020-03-19T21:18:16.594 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.592+0000 7f8d7c1df700 1 -- 172.21.15.170:0/3574523377 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8d74125840 msgr2=0x7f8d7411e070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:16.595 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.592+0000 7f8d7c1df700 1 --2- 172.21.15.170:0/3574523377 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8d74125840 0x7f8d7411e070 secure :-1 s=READY pgs=121 cs=0 l=1 rx=0x7f8d74065c90 tx=0x7f8d70007a40).stop 2020-03-19T21:18:16.595 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.592+0000 7f8d7c1df700 1 -- 172.21.15.170:0/3574523377 shutdown_connections 2020-03-19T21:18:16.595 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.592+0000 7f8d7c1df700 1 --2- 172.21.15.170:0/3574523377 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f8d54028280 0x7f8d5402a740 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:16.595 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.592+0000 7f8d7c1df700 1 --2- 172.21.15.170:0/3574523377 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8d74125840 0x7f8d7411e070 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:16.595 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.592+0000 7f8d7c1df700 1 -- 172.21.15.170:0/3574523377 >> 172.21.15.170:0/3574523377 conn(0x7f8d74091390 msgr2=0x7f8d7411f220 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:16.595 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.592+0000 7f8d7c1df700 1 -- 172.21.15.170:0/3574523377 shutdown_connections 2020-03-19T21:18:16.595 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.592+0000 7f8d7c1df700 1 -- 172.21.15.170:0/3574523377 wait complete. 2020-03-19T21:18:16.604 INFO:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2020-03-19T21:18:16.711 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.709+0000 7f1d6c26e700 1 Processor -- start 2020-03-19T21:18:16.711 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.709+0000 7f1d6c26e700 1 -- start start 2020-03-19T21:18:16.716 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.709+0000 7f1d6c26e700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f1d641224c0 0x7f1d64122ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T21:18:16.716 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.709+0000 7f1d6c26e700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d641228a0 con 0x7f1d64122f40 2020-03-19T21:18:16.717 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.709+0000 7f1d6c26e700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d64092200 con 0x7f1d641224c0 2020-03-19T21:18:16.717 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.710+0000 7f1d6a00a700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f1d641224c0 0x7f1d64122ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:16.717 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.710+0000 7f1d69809700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f1d64122f40 0x7f1d64060110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:53698/0 (socket says 172.21.15.170:53698) 2020-03-19T21:18:16.717 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.710+0000 7f1d6a00a700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f1d641224c0 0x7f1d64122ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:52934/0 (socket says 172.21.15.170:52934) 2020-03-19T21:18:16.717 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.710+0000 7f1d69809700 1 -- 172.21.15.170:0/4057507455 learned_addr learned my addr 172.21.15.170:0/4057507455 (peer_addr_for_me v1:172.21.15.170:0/0) 2020-03-19T21:18:16.717 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.710+0000 7f1d6a00a700 1 -- 172.21.15.170:0/4057507455 >> v1:172.21.15.170:6789/0 conn(0x7f1d64122f40 legacy=0x7f1d64060110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:16.718 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.710+0000 7f1d6a00a700 1 -- 172.21.15.170:0/4057507455 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d64090b60 con 0x7f1d641224c0 2020-03-19T21:18:16.718 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.710+0000 7f1d6a00a700 1 --2- 172.21.15.170:0/4057507455 >> v2:172.21.15.170:3300/0 conn(0x7f1d641224c0 0x7f1d64122ae0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f1d5c0032b0 tx=0x7f1d5c007dc0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:16.718 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.710+0000 7f1d6a00a700 1 --2- 172.21.15.170:0/4057507455 >> v2:172.21.15.170:3300/0 conn(0x7f1d641224c0 0x7f1d64122ae0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f1d5c0032b0 tx=0x7f1d5c007dc0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:16.718 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.710+0000 7f1d6a00a700 1 --2- 172.21.15.170:0/4057507455 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d641224c0 0x7f1d64122ae0 secure :-1 s=READY pgs=123 cs=0 l=1 rx=0x7f1d5c0032b0 tx=0x7f1d5c007dc0).ready entity=mon.0 client_cookie=b0aaf7e00e460928 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:16.718 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.710+0000 7f1d6a00a700 1 --2- 172.21.15.170:0/4057507455 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d641224c0 0x7f1d64122ae0 secure :-1 s=THROTTLE_DONE pgs=123 cs=0 l=1 rx=0x7f1d5c0032b0 tx=0x7f1d5c007dc0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:16.718 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.710+0000 7f1d69008700 1 -- 172.21.15.170:0/4057507455 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f1d5c011cd0 con 0x7f1d641224c0 2020-03-19T21:18:16.719 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.710+0000 7f1d6a00a700 1 --2- 172.21.15.170:0/4057507455 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d641224c0 0x7f1d64122ae0 secure :-1 s=THROTTLE_DONE pgs=123 cs=0 l=1 rx=0x7f1d5c0032b0 tx=0x7f1d5c007dc0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:16.719 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.710+0000 7f1d6a00a700 1 --2- 172.21.15.170:0/4057507455 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d641224c0 0x7f1d64122ae0 secure :-1 s=THROTTLE_DONE pgs=123 cs=0 l=1 rx=0x7f1d5c0032b0 tx=0x7f1d5c007dc0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:16.719 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.710+0000 7f1d69008700 1 -- 172.21.15.170:0/4057507455 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d5c011e30 con 0x7f1d641224c0 2020-03-19T21:18:16.719 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.710+0000 7f1d69008700 1 -- 172.21.15.170:0/4057507455 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f1d5c00a290 con 0x7f1d641224c0 2020-03-19T21:18:16.719 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.710+0000 7f1d6c26e700 1 -- 172.21.15.170:0/4057507455 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d641224c0 msgr2=0x7f1d64122ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:16.719 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.710+0000 7f1d6c26e700 1 --2- 172.21.15.170:0/4057507455 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d641224c0 0x7f1d64122ae0 secure :-1 s=READY pgs=123 cs=0 l=1 rx=0x7f1d5c0032b0 tx=0x7f1d5c007dc0).stop 2020-03-19T21:18:16.719 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.711+0000 7f1d6c26e700 1 -- 172.21.15.170:0/4057507455 shutdown_connections 2020-03-19T21:18:16.720 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.711+0000 7f1d6c26e700 1 --2- 172.21.15.170:0/4057507455 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d641224c0 0x7f1d64122ae0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:16.720 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.711+0000 7f1d6c26e700 1 -- 172.21.15.170:0/4057507455 >> 172.21.15.170:0/4057507455 conn(0x7f1d64091390 msgr2=0x7f1d64091770 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:16.720 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.711+0000 7f1d6c26e700 1 -- 172.21.15.170:0/4057507455 shutdown_connections 2020-03-19T21:18:16.720 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.711+0000 7f1d6c26e700 1 -- 172.21.15.170:0/4057507455 wait complete. 2020-03-19T21:18:16.720 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.711+0000 7f1d6c26e700 1 Processor -- start 2020-03-19T21:18:16.720 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.711+0000 7f1d6c26e700 1 -- start start 2020-03-19T21:18:16.720 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.712+0000 7f1d6c26e700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f1d64122f40 0x7f1d6410b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:16.720 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.712+0000 7f1d6c26e700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d64123620 con 0x7f1d641224c0 2020-03-19T21:18:16.721 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.712+0000 7f1d6c26e700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d6410d9f0 con 0x7f1d64122f40 2020-03-19T21:18:16.721 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.712+0000 7f1d6a00a700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f1d641224c0 0x7f1d6412fe90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:53702/0 (socket says 172.21.15.170:53702) 2020-03-19T21:18:16.721 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.712+0000 7f1d6a00a700 1 -- 172.21.15.170:0/3387029074 learned_addr learned my addr 172.21.15.170:0/3387029074 (peer_addr_for_me v1:172.21.15.170:0/0) 2020-03-19T21:18:16.721 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.712+0000 7f1d69809700 1 --2- 172.21.15.170:0/3387029074 >> v2:172.21.15.170:3300/0 conn(0x7f1d64122f40 0x7f1d6410b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:16.721 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.712+0000 7f1d69809700 1 -- 172.21.15.170:0/3387029074 >> v1:172.21.15.170:6789/0 conn(0x7f1d641224c0 legacy=0x7f1d6412fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:16.721 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.712+0000 7f1d577fe700 1 -- 172.21.15.170:0/3387029074 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2570616958 0 0) 0x7f1d64123620 con 0x7f1d641224c0 2020-03-19T21:18:16.721 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.712+0000 7f1d69809700 1 -- 172.21.15.170:0/3387029074 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d64123bd0 con 0x7f1d64122f40 2020-03-19T21:18:16.722 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.712+0000 7f1d69809700 1 --2- 172.21.15.170:0/3387029074 >> v2:172.21.15.170:3300/0 conn(0x7f1d64122f40 0x7f1d6410b400 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f1d6408e750 tx=0x7f1d580108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:16.722 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.712+0000 7f1d69809700 1 --2- 172.21.15.170:0/3387029074 >> v2:172.21.15.170:3300/0 conn(0x7f1d64122f40 0x7f1d6410b400 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f1d6408e750 tx=0x7f1d580108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:16.722 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.712+0000 7f1d69809700 1 --2- 172.21.15.170:0/3387029074 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d64122f40 0x7f1d6410b400 secure :-1 s=READY pgs=125 cs=0 l=1 rx=0x7f1d6408e750 tx=0x7f1d580108a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:16.722 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.712+0000 7f1d69809700 1 --2- 172.21.15.170:0/3387029074 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d64122f40 0x7f1d6410b400 secure :-1 s=THROTTLE_DONE pgs=125 cs=0 l=1 rx=0x7f1d6408e750 tx=0x7f1d580108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:16.722 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.712+0000 7f1d577fe700 1 -- 172.21.15.170:0/3387029074 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f1d58005b20 con 0x7f1d64122f40 2020-03-19T21:18:16.722 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.712+0000 7f1d69809700 1 --2- 172.21.15.170:0/3387029074 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d64122f40 0x7f1d6410b400 secure :-1 s=THROTTLE_DONE pgs=125 cs=0 l=1 rx=0x7f1d6408e750 tx=0x7f1d580108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:16.722 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.712+0000 7f1d69809700 1 --2- 172.21.15.170:0/3387029074 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d64122f40 0x7f1d6410b400 secure :-1 s=THROTTLE_DONE pgs=125 cs=0 l=1 rx=0x7f1d6408e750 tx=0x7f1d580108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:16.723 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.712+0000 7f1d577fe700 1 -- 172.21.15.170:0/3387029074 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d58005c80 con 0x7f1d64122f40 2020-03-19T21:18:16.723 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.712+0000 7f1d6c26e700 1 -- 172.21.15.170:0/3387029074 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d641238c0 con 0x7f1d64122f40 2020-03-19T21:18:16.723 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.712+0000 7f1d577fe700 1 -- 172.21.15.170:0/3387029074 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f1d580060c0 con 0x7f1d64122f40 2020-03-19T21:18:16.723 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.712+0000 7f1d6c26e700 1 -- 172.21.15.170:0/3387029074 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d6411fce0 con 0x7f1d64122f40 2020-03-19T21:18:16.723 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.713+0000 7f1d69809700 1 --2- 172.21.15.170:0/3387029074 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d64122f40 0x7f1d6410b400 secure :-1 s=THROTTLE_DONE pgs=125 cs=0 l=1 rx=0x7f1d6408e750 tx=0x7f1d580108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:16.723 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.713+0000 7f1d577fe700 1 -- 172.21.15.170:0/3387029074 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 4) v1 ==== 29603+0+0 (secure 0 0 0) 0x7f1d5800a320 con 0x7f1d64122f40 2020-03-19T21:18:16.723 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.713+0000 7f1d69809700 1 --2- 172.21.15.170:0/3387029074 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d64122f40 0x7f1d6410b400 secure :-1 s=THROTTLE_DONE pgs=125 cs=0 l=1 rx=0x7f1d6408e750 tx=0x7f1d580108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:16.724 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.713+0000 7f1d577fe700 1 --2- 172.21.15.170:0/3387029074 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f1d44028230 0x7f1d4402a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:16.724 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.713+0000 7f1d577fe700 1 -- 172.21.15.170:0/3387029074 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3403+0+0 (secure 0 0 0) 0x7f1d58033f20 con 0x7f1d64122f40 2020-03-19T21:18:16.724 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.713+0000 7f1d6a00a700 1 --2- 172.21.15.170:0/3387029074 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f1d44028230 0x7f1d4402a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:16.725 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.714+0000 7f1d6a00a700 1 --2- 172.21.15.170:0/3387029074 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f1d44028230 0x7f1d4402a6f0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f1d5c010540 tx=0x7f1d5c007dc0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:16.725 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.714+0000 7f1d6a00a700 1 --2- 172.21.15.170:0/3387029074 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f1d44028230 0x7f1d4402a6f0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f1d5c010540 tx=0x7f1d5c007dc0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:16.725 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.714+0000 7f1d6a00a700 1 --2- 172.21.15.170:0/3387029074 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f1d44028230 0x7f1d4402a6f0 secure :-1 s=READY pgs=31 cs=0 l=1 rx=0x7f1d5c010540 tx=0x7f1d5c007dc0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:16.725 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.714+0000 7f1d6c26e700 1 -- 172.21.15.170:0/3387029074 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d64130760 con 0x7f1d64122f40 2020-03-19T21:18:16.725 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.717+0000 7f1d69809700 1 --2- 172.21.15.170:0/3387029074 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d64122f40 0x7f1d6410b400 secure :-1 s=THROTTLE_DONE pgs=125 cs=0 l=1 rx=0x7f1d6408e750 tx=0x7f1d580108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:16.726 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.717+0000 7f1d577fe700 1 -- 172.21.15.170:0/3387029074 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+126772 (secure 0 0 0) 0x7f1d64130760 con 0x7f1d64122f40 2020-03-19T21:18:16.861 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.859+0000 7f1d6c26e700 1 -- 172.21.15.170:0/3387029074 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1d64130760 con 0x7f1d64122f40 2020-03-19T21:18:16.862 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.860+0000 7f1d69809700 1 --2- 172.21.15.170:0/3387029074 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d64122f40 0x7f1d6410b400 secure :-1 s=THROTTLE_DONE pgs=125 cs=0 l=1 rx=0x7f1d6408e750 tx=0x7f1d580108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:16.862 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.860+0000 7f1d577fe700 1 -- 172.21.15.170:0/3387029074 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v16) v1 ==== 100+0+10 (secure 0 0 0) 0x7f1d64130760 con 0x7f1d64122f40 2020-03-19T21:18:16.862 INFO:teuthology.orchestra.run.smithi170.stdout:pg_num: 8 2020-03-19T21:18:16.863 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.861+0000 7f1d6c26e700 1 -- 172.21.15.170:0/3387029074 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f1d44028230 msgr2=0x7f1d4402a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:16.863 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.861+0000 7f1d6c26e700 1 --2- 172.21.15.170:0/3387029074 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f1d44028230 0x7f1d4402a6f0 secure :-1 s=READY pgs=31 cs=0 l=1 rx=0x7f1d5c010540 tx=0x7f1d5c007dc0).stop 2020-03-19T21:18:16.864 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.861+0000 7f1d6c26e700 1 -- 172.21.15.170:0/3387029074 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d64122f40 msgr2=0x7f1d6410b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:16.864 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.861+0000 7f1d6c26e700 1 --2- 172.21.15.170:0/3387029074 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d64122f40 0x7f1d6410b400 secure :-1 s=READY pgs=125 cs=0 l=1 rx=0x7f1d6408e750 tx=0x7f1d580108a0).stop 2020-03-19T21:18:16.864 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.861+0000 7f1d6c26e700 1 -- 172.21.15.170:0/3387029074 shutdown_connections 2020-03-19T21:18:16.864 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.861+0000 7f1d6c26e700 1 --2- 172.21.15.170:0/3387029074 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f1d44028230 0x7f1d4402a6f0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:16.864 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.861+0000 7f1d6c26e700 1 --2- 172.21.15.170:0/3387029074 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d64122f40 0x7f1d6410b400 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:16.864 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.861+0000 7f1d6c26e700 1 -- 172.21.15.170:0/3387029074 >> 172.21.15.170:0/3387029074 conn(0x7f1d64091390 msgr2=0x7f1d64093e00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:16.864 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.861+0000 7f1d6c26e700 1 -- 172.21.15.170:0/3387029074 shutdown_connections 2020-03-19T21:18:16.865 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.861+0000 7f1d6c26e700 1 -- 172.21.15.170:0/3387029074 wait complete. 2020-03-19T21:18:16.873 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2020-03-19T21:18:16.873 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2020-03-19T21:18:16.874 INFO:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early mgr dump --format=json 2020-03-19T21:18:16.979 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.977+0000 7fd8fca0a700 1 Processor -- start 2020-03-19T21:18:16.979 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.977+0000 7fd8fca0a700 1 -- start start 2020-03-19T21:18:16.985 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.978+0000 7fd8fca0a700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fd8f80907c0 0x7fd8f808daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T21:18:16.985 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.978+0000 7fd8fca0a700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8f80605e0 con 0x7fd8f808d000 2020-03-19T21:18:16.985 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.978+0000 7fd8fca0a700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8f808ef70 con 0x7fd8f80907c0 2020-03-19T21:18:16.985 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.978+0000 7fd8f5d9b700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fd8f80907c0 0x7fd8f808daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:16.985 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.978+0000 7fd8f5d9b700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fd8f80907c0 0x7fd8f808daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:52946/0 (socket says 172.21.15.170:52946) 2020-03-19T21:18:16.985 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.978+0000 7fd8f659c700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fd8f808d000 0x7fd8f808d3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:53708/0 (socket says 172.21.15.170:53708) 2020-03-19T21:18:16.986 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.978+0000 7fd8f5d9b700 1 -- 172.21.15.170:0/1523729982 learned_addr learned my addr 172.21.15.170:0/1523729982 (peer_addr_for_me v2:172.21.15.170:0/0) 2020-03-19T21:18:16.986 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.978+0000 7fd8f5d9b700 1 -- 172.21.15.170:0/1523729982 >> v1:172.21.15.170:6789/0 conn(0x7fd8f808d000 legacy=0x7fd8f808d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:16.986 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.978+0000 7fd8f559a700 1 -- 172.21.15.170:0/1523729982 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3709111314 0 0) 0x7fd8f80605e0 con 0x7fd8f808d000 2020-03-19T21:18:16.986 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.978+0000 7fd8f5d9b700 1 -- 172.21.15.170:0/1523729982 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8f8060810 con 0x7fd8f80907c0 2020-03-19T21:18:16.986 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.978+0000 7fd8f5d9b700 1 --2- 172.21.15.170:0/1523729982 >> v2:172.21.15.170:3300/0 conn(0x7fd8f80907c0 0x7fd8f808daf0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7fd8f8090c70 tx=0x7fd8e80088e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:16.986 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.978+0000 7fd8f5d9b700 1 --2- 172.21.15.170:0/1523729982 >> v2:172.21.15.170:3300/0 conn(0x7fd8f80907c0 0x7fd8f808daf0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7fd8f8090c70 tx=0x7fd8e80088e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:16.986 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.978+0000 7fd8f5d9b700 1 --2- 172.21.15.170:0/1523729982 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd8f80907c0 0x7fd8f808daf0 secure :-1 s=READY pgs=127 cs=0 l=1 rx=0x7fd8f8090c70 tx=0x7fd8e80088e0).ready entity=mon.0 client_cookie=e6c45ff731e2e13e server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:16.987 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.978+0000 7fd8f5d9b700 1 --2- 172.21.15.170:0/1523729982 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd8f80907c0 0x7fd8f808daf0 secure :-1 s=THROTTLE_DONE pgs=127 cs=0 l=1 rx=0x7fd8f8090c70 tx=0x7fd8e80088e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:16.987 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.978+0000 7fd8f559a700 1 -- 172.21.15.170:0/1523729982 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fd8e80114b0 con 0x7fd8f80907c0 2020-03-19T21:18:16.987 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.978+0000 7fd8f5d9b700 1 --2- 172.21.15.170:0/1523729982 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd8f80907c0 0x7fd8f808daf0 secure :-1 s=THROTTLE_DONE pgs=127 cs=0 l=1 rx=0x7fd8f8090c70 tx=0x7fd8e80088e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:16.987 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.978+0000 7fd8f559a700 1 -- 172.21.15.170:0/1523729982 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8e8011610 con 0x7fd8f80907c0 2020-03-19T21:18:16.987 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.978+0000 7fd8f5d9b700 1 --2- 172.21.15.170:0/1523729982 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd8f80907c0 0x7fd8f808daf0 secure :-1 s=THROTTLE_DONE pgs=127 cs=0 l=1 rx=0x7fd8f8090c70 tx=0x7fd8e80088e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:16.987 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.978+0000 7fd8fca0a700 1 -- 172.21.15.170:0/1523729982 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd8f80907c0 msgr2=0x7fd8f808daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:16.987 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.978+0000 7fd8f559a700 1 -- 172.21.15.170:0/1523729982 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fd8e80118b0 con 0x7fd8f80907c0 2020-03-19T21:18:16.988 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.978+0000 7fd8fca0a700 1 --2- 172.21.15.170:0/1523729982 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd8f80907c0 0x7fd8f808daf0 secure :-1 s=READY pgs=127 cs=0 l=1 rx=0x7fd8f8090c70 tx=0x7fd8e80088e0).stop 2020-03-19T21:18:16.988 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.979+0000 7fd8fca0a700 1 -- 172.21.15.170:0/1523729982 shutdown_connections 2020-03-19T21:18:16.988 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.979+0000 7fd8fca0a700 1 --2- 172.21.15.170:0/1523729982 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd8f80907c0 0x7fd8f808daf0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:16.988 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.979+0000 7fd8fca0a700 1 -- 172.21.15.170:0/1523729982 >> 172.21.15.170:0/1523729982 conn(0x7fd8f8091540 msgr2=0x7fd8f8091950 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:16.988 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.979+0000 7fd8fca0a700 1 -- 172.21.15.170:0/1523729982 shutdown_connections 2020-03-19T21:18:16.988 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.979+0000 7fd8fca0a700 1 -- 172.21.15.170:0/1523729982 wait complete. 2020-03-19T21:18:16.988 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.980+0000 7fd8fca0a700 1 Processor -- start 2020-03-19T21:18:16.989 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.980+0000 7fd8fca0a700 1 -- start start 2020-03-19T21:18:16.989 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.980+0000 7fd8fca0a700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fd8f808d000 0x7fd8f811a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:16.989 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.980+0000 7fd8fca0a700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8f8123820 con 0x7fd8f80907c0 2020-03-19T21:18:16.989 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.980+0000 7fd8fca0a700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8f8064a50 con 0x7fd8f808d000 2020-03-19T21:18:16.989 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.980+0000 7fd8f659c700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fd8f808d000 0x7fd8f811a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:16.989 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.980+0000 7fd8f5d9b700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fd8f80907c0 0x7fd8f8065110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:53714/0 (socket says 172.21.15.170:53714) 2020-03-19T21:18:16.989 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.980+0000 7fd8f5d9b700 1 -- 172.21.15.170:0/3121818993 learned_addr learned my addr 172.21.15.170:0/3121818993 (peer_addr_for_me v1:172.21.15.170:0/0) 2020-03-19T21:18:16.990 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.980+0000 7fd8f659c700 1 -- 172.21.15.170:0/3121818993 >> v1:172.21.15.170:6789/0 conn(0x7fd8f80907c0 legacy=0x7fd8f8065110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:16.990 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.980+0000 7fd8e77fe700 1 -- 172.21.15.170:0/3121818993 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 74706413 0 0) 0x7fd8f8123820 con 0x7fd8f80907c0 2020-03-19T21:18:16.990 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.980+0000 7fd8f659c700 1 -- 172.21.15.170:0/3121818993 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8f811aa30 con 0x7fd8f808d000 2020-03-19T21:18:16.990 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.980+0000 7fd8f659c700 1 --2- 172.21.15.170:0/3121818993 >> v2:172.21.15.170:3300/0 conn(0x7fd8f808d000 0x7fd8f811a5d0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7fd8ec000b80 tx=0x7fd8ec0108f0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:16.990 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.980+0000 7fd8f659c700 1 --2- 172.21.15.170:0/3121818993 >> v2:172.21.15.170:3300/0 conn(0x7fd8f808d000 0x7fd8f811a5d0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7fd8ec000b80 tx=0x7fd8ec0108f0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:16.990 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.980+0000 7fd8f659c700 1 --2- 172.21.15.170:0/3121818993 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd8f808d000 0x7fd8f811a5d0 secure :-1 s=READY pgs=129 cs=0 l=1 rx=0x7fd8ec000b80 tx=0x7fd8ec0108f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:16.990 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.981+0000 7fd8f659c700 1 --2- 172.21.15.170:0/3121818993 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd8f808d000 0x7fd8f811a5d0 secure :-1 s=THROTTLE_DONE pgs=129 cs=0 l=1 rx=0x7fd8ec000b80 tx=0x7fd8ec0108f0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:16.991 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.981+0000 7fd8e77fe700 1 -- 172.21.15.170:0/3121818993 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fd8ec011480 con 0x7fd8f808d000 2020-03-19T21:18:16.991 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.981+0000 7fd8f659c700 1 --2- 172.21.15.170:0/3121818993 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd8f808d000 0x7fd8f811a5d0 secure :-1 s=THROTTLE_DONE pgs=129 cs=0 l=1 rx=0x7fd8ec000b80 tx=0x7fd8ec0108f0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:16.991 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.981+0000 7fd8e77fe700 1 -- 172.21.15.170:0/3121818993 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8ec0115e0 con 0x7fd8f808d000 2020-03-19T21:18:16.991 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.981+0000 7fd8f659c700 1 --2- 172.21.15.170:0/3121818993 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd8f808d000 0x7fd8f811a5d0 secure :-1 s=THROTTLE_DONE pgs=129 cs=0 l=1 rx=0x7fd8ec000b80 tx=0x7fd8ec0108f0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:16.992 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.981+0000 7fd8fca0a700 1 -- 172.21.15.170:0/3121818993 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8f8066220 con 0x7fd8f808d000 2020-03-19T21:18:16.992 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.981+0000 7fd8e77fe700 1 -- 172.21.15.170:0/3121818993 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fd8ec011940 con 0x7fd8f808d000 2020-03-19T21:18:16.993 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.981+0000 7fd8fca0a700 1 -- 172.21.15.170:0/3121818993 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8f811a310 con 0x7fd8f808d000 2020-03-19T21:18:16.993 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.981+0000 7fd8f659c700 1 --2- 172.21.15.170:0/3121818993 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd8f808d000 0x7fd8f811a5d0 secure :-1 s=THROTTLE_DONE pgs=129 cs=0 l=1 rx=0x7fd8ec000b80 tx=0x7fd8ec0108f0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:16.993 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.981+0000 7fd8e77fe700 1 -- 172.21.15.170:0/3121818993 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 4) v1 ==== 29603+0+0 (secure 0 0 0) 0x7fd8ec00b0a0 con 0x7fd8f808d000 2020-03-19T21:18:16.993 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.981+0000 7fd8f659c700 1 --2- 172.21.15.170:0/3121818993 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd8f808d000 0x7fd8f811a5d0 secure :-1 s=THROTTLE_DONE pgs=129 cs=0 l=1 rx=0x7fd8ec000b80 tx=0x7fd8ec0108f0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:16.993 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.981+0000 7fd8e77fe700 1 --2- 172.21.15.170:0/3121818993 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7fd8d40281b0 0x7fd8d402a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:16.993 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.981+0000 7fd8e77fe700 1 -- 172.21.15.170:0/3121818993 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3403+0+0 (secure 0 0 0) 0x7fd8ec033fe0 con 0x7fd8f808d000 2020-03-19T21:18:16.993 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.982+0000 7fd8f5d9b700 1 --2- 172.21.15.170:0/3121818993 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7fd8d40281b0 0x7fd8d402a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:16.994 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.982+0000 7fd8f5d9b700 1 --2- 172.21.15.170:0/3121818993 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7fd8d40281b0 0x7fd8d402a670 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7fd8e8010280 tx=0x7fd8f8067850).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:16.994 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.982+0000 7fd8f5d9b700 1 --2- 172.21.15.170:0/3121818993 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7fd8d40281b0 0x7fd8d402a670 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7fd8e8010280 tx=0x7fd8f8067850).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:16.994 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.982+0000 7fd8f5d9b700 1 --2- 172.21.15.170:0/3121818993 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7fd8d40281b0 0x7fd8d402a670 secure :-1 s=READY pgs=32 cs=0 l=1 rx=0x7fd8e8010280 tx=0x7fd8f8067850).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:16.994 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.982+0000 7fd8fca0a700 1 -- 172.21.15.170:0/3121818993 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8f811b4f0 con 0x7fd8f808d000 2020-03-19T21:18:16.995 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.985+0000 7fd8f659c700 1 --2- 172.21.15.170:0/3121818993 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd8f808d000 0x7fd8f811a5d0 secure :-1 s=THROTTLE_DONE pgs=129 cs=0 l=1 rx=0x7fd8ec000b80 tx=0x7fd8ec0108f0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:16.996 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:16.985+0000 7fd8e77fe700 1 -- 172.21.15.170:0/3121818993 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+126772 (secure 0 0 0) 0x7fd8ec011d90 con 0x7fd8f808d000 2020-03-19T21:18:17.161 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.159+0000 7fd8fca0a700 1 -- 172.21.15.170:0/3121818993 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fd8f811b0e0 con 0x7fd8f808d000 2020-03-19T21:18:17.162 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.160+0000 7fd8f659c700 1 --2- 172.21.15.170:0/3121818993 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd8f808d000 0x7fd8f811a5d0 secure :-1 s=THROTTLE_DONE pgs=129 cs=0 l=1 rx=0x7fd8ec000b80 tx=0x7fd8ec0108f0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:17.162 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.160+0000 7fd8e77fe700 1 -- 172.21.15.170:0/3121818993 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v4) v1 ==== 74+0+60001 (secure 0 0 0) 0x7fd8ec007160 con 0x7fd8f808d000 2020-03-19T21:18:17.163 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:18:17.164 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.162+0000 7fd8fca0a700 1 -- 172.21.15.170:0/3121818993 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7fd8d40281b0 msgr2=0x7fd8d402a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:17.165 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.162+0000 7fd8fca0a700 1 --2- 172.21.15.170:0/3121818993 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7fd8d40281b0 0x7fd8d402a670 secure :-1 s=READY pgs=32 cs=0 l=1 rx=0x7fd8e8010280 tx=0x7fd8f8067850).stop 2020-03-19T21:18:17.165 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.162+0000 7fd8fca0a700 1 -- 172.21.15.170:0/3121818993 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd8f808d000 msgr2=0x7fd8f811a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:17.165 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.162+0000 7fd8fca0a700 1 --2- 172.21.15.170:0/3121818993 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd8f808d000 0x7fd8f811a5d0 secure :-1 s=READY pgs=129 cs=0 l=1 rx=0x7fd8ec000b80 tx=0x7fd8ec0108f0).stop 2020-03-19T21:18:17.165 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.162+0000 7fd8fca0a700 1 -- 172.21.15.170:0/3121818993 shutdown_connections 2020-03-19T21:18:17.165 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.162+0000 7fd8fca0a700 1 --2- 172.21.15.170:0/3121818993 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7fd8d40281b0 0x7fd8d402a670 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:17.166 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.162+0000 7fd8fca0a700 1 --2- 172.21.15.170:0/3121818993 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd8f808d000 0x7fd8f811a5d0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:17.166 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.162+0000 7fd8fca0a700 1 -- 172.21.15.170:0/3121818993 >> 172.21.15.170:0/3121818993 conn(0x7fd8f8091540 msgr2=0x7fd8f8125db0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:17.166 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.162+0000 7fd8fca0a700 1 -- 172.21.15.170:0/3121818993 shutdown_connections 2020-03-19T21:18:17.166 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.162+0000 7fd8fca0a700 1 -- 172.21.15.170:0/3121818993 wait complete. 2020-03-19T21:18:17.175 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":4,"active_gid":4102,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":20154},{"type":"v1","addr":"172.21.15.170:6825","nonce":20154}]},"active_addr":"172.21.15.170:6825/20154","active_change":"2020-03-19T21:18:02.504416+0000","active_mgr_features":4540138292836696063,"available":true,"standbys":[],"modules":["iostat","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"none","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_cephfs":{"name":"FEATURE_TOGGLE_cephfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_iscsi":{"name":"FEATURE_TOGGLE_iscsi","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_mirroring":{"name":"FEATURE_TOGGLE_mirroring","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_nfs":{"name":"FEATURE_TOGGLE_nfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rbd":{"name":"FEATURE_TOGGLE_rbd","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rgw":{"name":"FEATURE_TOGGLE_rgw","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_HOST":{"name":"RGW_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_PORT":{"name":"RGW_API_PORT","type":"int","level":"advanced","flags":0,"default_value":"80","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SCHEME":{"name":"RGW_API_SCHEME","type":"str","level":"advanced","flags":0,"default_value":"http","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_USER_ID":{"name":"RGW_API_USER_ID","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"None","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"persist_interval":{"name":"persist_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how frequently to persist completed events","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"str","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"nautilus":["balancer","crash","devicehealth","orchestrator_cli","progress","rbd_support","status","volumes"],"octopus":["balancer","crash","devicehealth","orchestrator","osd_support","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"last_failure_osd_epoch":0,"active_clients":[{"addrvec":[{"type":"v2","addr":"172.21.15.170:0","nonce":3912883030}]},{"addrvec":[{"type":"v2","addr":"172.21.15.170:0","nonce":2476957820}]}]}} 2020-03-19T21:18:17.179 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2020-03-19T21:18:17.179 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2020-03-19T21:18:17.180 INFO:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-03-19T21:18:17.285 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.283+0000 7f6644991700 1 Processor -- start 2020-03-19T21:18:17.285 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.283+0000 7f6644991700 1 -- start start 2020-03-19T21:18:17.290 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.284+0000 7f6644991700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f66400907c0 0x7f664008daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T21:18:17.290 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.284+0000 7f6644991700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66400605e0 con 0x7f664008d000 2020-03-19T21:18:17.291 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.284+0000 7f6644991700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f664008ef70 con 0x7f66400907c0 2020-03-19T21:18:17.291 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.284+0000 7f663dd9b700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f66400907c0 0x7f664008daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:17.291 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.284+0000 7f663e59c700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f664008d000 0x7f664008d3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:53718/0 (socket says 172.21.15.170:53718) 2020-03-19T21:18:17.291 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.284+0000 7f663dd9b700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f66400907c0 0x7f664008daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:52956/0 (socket says 172.21.15.170:52956) 2020-03-19T21:18:17.291 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.284+0000 7f663e59c700 1 -- 172.21.15.170:0/3849730656 learned_addr learned my addr 172.21.15.170:0/3849730656 (peer_addr_for_me v1:172.21.15.170:0/0) 2020-03-19T21:18:17.291 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.284+0000 7f663dd9b700 1 -- 172.21.15.170:0/3849730656 >> v1:172.21.15.170:6789/0 conn(0x7f664008d000 legacy=0x7f664008d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:17.291 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.284+0000 7f663dd9b700 1 -- 172.21.15.170:0/3849730656 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6640060810 con 0x7f66400907c0 2020-03-19T21:18:17.292 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.284+0000 7f663d59a700 1 -- 172.21.15.170:0/3849730656 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2726848509 0 0) 0x7f66400605e0 con 0x7f664008d000 2020-03-19T21:18:17.292 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.284+0000 7f663dd9b700 1 --2- 172.21.15.170:0/3849730656 >> v2:172.21.15.170:3300/0 conn(0x7f66400907c0 0x7f664008daf0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f6640090c70 tx=0x7f66300088e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:17.292 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.284+0000 7f663dd9b700 1 --2- 172.21.15.170:0/3849730656 >> v2:172.21.15.170:3300/0 conn(0x7f66400907c0 0x7f664008daf0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f6640090c70 tx=0x7f66300088e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:17.292 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.284+0000 7f663dd9b700 1 --2- 172.21.15.170:0/3849730656 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f66400907c0 0x7f664008daf0 secure :-1 s=READY pgs=131 cs=0 l=1 rx=0x7f6640090c70 tx=0x7f66300088e0).ready entity=mon.0 client_cookie=9c13d36442e9f431 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:17.292 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.284+0000 7f663dd9b700 1 --2- 172.21.15.170:0/3849730656 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f66400907c0 0x7f664008daf0 secure :-1 s=THROTTLE_DONE pgs=131 cs=0 l=1 rx=0x7f6640090c70 tx=0x7f66300088e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:17.292 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.284+0000 7f663d59a700 1 -- 172.21.15.170:0/3849730656 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f66300113f0 con 0x7f66400907c0 2020-03-19T21:18:17.292 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.284+0000 7f663dd9b700 1 --2- 172.21.15.170:0/3849730656 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f66400907c0 0x7f664008daf0 secure :-1 s=THROTTLE_DONE pgs=131 cs=0 l=1 rx=0x7f6640090c70 tx=0x7f66300088e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:17.292 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.284+0000 7f663d59a700 1 -- 172.21.15.170:0/3849730656 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6630011550 con 0x7f66400907c0 2020-03-19T21:18:17.293 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.284+0000 7f663dd9b700 1 --2- 172.21.15.170:0/3849730656 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f66400907c0 0x7f664008daf0 secure :-1 s=THROTTLE_DONE pgs=131 cs=0 l=1 rx=0x7f6640090c70 tx=0x7f66300088e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:17.293 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.284+0000 7f663d59a700 1 -- 172.21.15.170:0/3849730656 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f6630011830 con 0x7f66400907c0 2020-03-19T21:18:17.293 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.284+0000 7f6644991700 1 -- 172.21.15.170:0/3849730656 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f66400907c0 msgr2=0x7f664008daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:17.293 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.284+0000 7f6644991700 1 --2- 172.21.15.170:0/3849730656 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f66400907c0 0x7f664008daf0 secure :-1 s=READY pgs=131 cs=0 l=1 rx=0x7f6640090c70 tx=0x7f66300088e0).stop 2020-03-19T21:18:17.293 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.285+0000 7f6644991700 1 -- 172.21.15.170:0/3849730656 shutdown_connections 2020-03-19T21:18:17.293 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.285+0000 7f6644991700 1 --2- 172.21.15.170:0/3849730656 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f66400907c0 0x7f664008daf0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:17.293 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.285+0000 7f6644991700 1 -- 172.21.15.170:0/3849730656 >> 172.21.15.170:0/3849730656 conn(0x7f6640091540 msgr2=0x7f6640091950 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:17.294 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.285+0000 7f6644991700 1 -- 172.21.15.170:0/3849730656 shutdown_connections 2020-03-19T21:18:17.294 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.285+0000 7f6644991700 1 -- 172.21.15.170:0/3849730656 wait complete. 2020-03-19T21:18:17.294 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.286+0000 7f6644991700 1 Processor -- start 2020-03-19T21:18:17.294 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.286+0000 7f6644991700 1 -- start start 2020-03-19T21:18:17.294 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.286+0000 7f6644991700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f66400907c0 0x7f66401228a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:17.294 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.286+0000 7f6644991700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f664011ff70 con 0x7f664008d000 2020-03-19T21:18:17.294 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.286+0000 7f6644991700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f664011fc60 con 0x7f66400907c0 2020-03-19T21:18:17.295 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.286+0000 7f663e59c700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f664008d000 0x7f6640122190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:53722/0 (socket says 172.21.15.170:53722) 2020-03-19T21:18:17.295 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.286+0000 7f663e59c700 1 -- 172.21.15.170:0/3058361741 learned_addr learned my addr 172.21.15.170:0/3058361741 (peer_addr_for_me v1:172.21.15.170:0/0) 2020-03-19T21:18:17.295 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.286+0000 7f663dd9b700 1 --2- 172.21.15.170:0/3058361741 >> v2:172.21.15.170:3300/0 conn(0x7f66400907c0 0x7f66401228a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:17.295 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.286+0000 7f662f7fe700 1 -- 172.21.15.170:0/3058361741 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1682445617 0 0) 0x7f664011ff70 con 0x7f664008d000 2020-03-19T21:18:17.295 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.286+0000 7f662f7fe700 1 -- 172.21.15.170:0/3058361741 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f661c002fe0 con 0x7f664008d000 2020-03-19T21:18:17.295 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.286+0000 7f663dd9b700 1 -- 172.21.15.170:0/3058361741 >> v1:172.21.15.170:6789/0 conn(0x7f664008d000 legacy=0x7f6640122190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:17.295 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.286+0000 7f663dd9b700 1 -- 172.21.15.170:0/3058361741 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f664011d270 con 0x7f66400907c0 2020-03-19T21:18:17.296 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.286+0000 7f663dd9b700 1 --2- 172.21.15.170:0/3058361741 >> v2:172.21.15.170:3300/0 conn(0x7f66400907c0 0x7f66401228a0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f6640090c70 tx=0x7f6640067850).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:17.296 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.286+0000 7f663dd9b700 1 --2- 172.21.15.170:0/3058361741 >> v2:172.21.15.170:3300/0 conn(0x7f66400907c0 0x7f66401228a0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f6640090c70 tx=0x7f6640067850).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:17.296 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.286+0000 7f663dd9b700 1 --2- 172.21.15.170:0/3058361741 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f66400907c0 0x7f66401228a0 secure :-1 s=READY pgs=133 cs=0 l=1 rx=0x7f6640090c70 tx=0x7f6640067850).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:17.296 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.287+0000 7f663dd9b700 1 --2- 172.21.15.170:0/3058361741 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f66400907c0 0x7f66401228a0 secure :-1 s=THROTTLE_DONE pgs=133 cs=0 l=1 rx=0x7f6640090c70 tx=0x7f6640067850).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:17.296 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.287+0000 7f662f7fe700 1 -- 172.21.15.170:0/3058361741 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f663000f6c0 con 0x7f66400907c0 2020-03-19T21:18:17.296 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.287+0000 7f663dd9b700 1 --2- 172.21.15.170:0/3058361741 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f66400907c0 0x7f66401228a0 secure :-1 s=THROTTLE_DONE pgs=133 cs=0 l=1 rx=0x7f6640090c70 tx=0x7f6640067850).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:17.296 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.287+0000 7f663dd9b700 1 --2- 172.21.15.170:0/3058361741 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f66400907c0 0x7f66401228a0 secure :-1 s=THROTTLE_DONE pgs=133 cs=0 l=1 rx=0x7f6640090c70 tx=0x7f6640067850).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:17.297 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.287+0000 7f662f7fe700 1 -- 172.21.15.170:0/3058361741 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f663000f820 con 0x7f66400907c0 2020-03-19T21:18:17.297 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.287+0000 7f662f7fe700 1 -- 172.21.15.170:0/3058361741 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f66300057f0 con 0x7f66400907c0 2020-03-19T21:18:17.297 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.287+0000 7f6644991700 1 -- 172.21.15.170:0/3058361741 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f664011f340 con 0x7f66400907c0 2020-03-19T21:18:17.297 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.287+0000 7f6644991700 1 -- 172.21.15.170:0/3058361741 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f664011e8f0 con 0x7f66400907c0 2020-03-19T21:18:17.297 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.287+0000 7f663dd9b700 1 --2- 172.21.15.170:0/3058361741 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f66400907c0 0x7f66401228a0 secure :-1 s=THROTTLE_DONE pgs=133 cs=0 l=1 rx=0x7f6640090c70 tx=0x7f6640067850).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:17.297 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.287+0000 7f662f7fe700 1 -- 172.21.15.170:0/3058361741 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 4) v1 ==== 29603+0+0 (secure 0 0 0) 0x7f6630005a60 con 0x7f66400907c0 2020-03-19T21:18:17.297 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.287+0000 7f663dd9b700 1 --2- 172.21.15.170:0/3058361741 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f66400907c0 0x7f66401228a0 secure :-1 s=THROTTLE_DONE pgs=133 cs=0 l=1 rx=0x7f6640090c70 tx=0x7f6640067850).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:17.297 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.287+0000 7f662f7fe700 1 --2- 172.21.15.170:0/3058361741 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f661c028760 0x7f661c02ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:17.298 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.287+0000 7f662f7fe700 1 -- 172.21.15.170:0/3058361741 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3403+0+0 (secure 0 0 0) 0x7f6630034750 con 0x7f66400907c0 2020-03-19T21:18:17.298 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.288+0000 7f663e59c700 1 --2- 172.21.15.170:0/3058361741 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f661c028760 0x7f661c02ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:17.298 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.288+0000 7f663e59c700 1 --2- 172.21.15.170:0/3058361741 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f661c028760 0x7f661c02ac20 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f6634005520 tx=0x7f66340016a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:17.298 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.288+0000 7f6644991700 1 -- 172.21.15.170:0/3058361741 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f664011fdd0 con 0x7f66400907c0 2020-03-19T21:18:17.298 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.288+0000 7f663e59c700 1 --2- 172.21.15.170:0/3058361741 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f661c028760 0x7f661c02ac20 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f6634005520 tx=0x7f66340016a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:17.298 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.288+0000 7f663e59c700 1 --2- 172.21.15.170:0/3058361741 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f661c028760 0x7f661c02ac20 secure :-1 s=READY pgs=33 cs=0 l=1 rx=0x7f6634005520 tx=0x7f66340016a0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:17.299 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.291+0000 7f663dd9b700 1 --2- 172.21.15.170:0/3058361741 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f66400907c0 0x7f66401228a0 secure :-1 s=THROTTLE_DONE pgs=133 cs=0 l=1 rx=0x7f6640090c70 tx=0x7f6640067850).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:17.299 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.291+0000 7f662f7fe700 1 -- 172.21.15.170:0/3058361741 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+126772 (secure 0 0 0) 0x7f664011fdd0 con 0x7f66400907c0 2020-03-19T21:18:17.431 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.429+0000 7f6644991700 1 -- 172.21.15.170:0/3058361741 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f664011fdd0 con 0x7f66400907c0 2020-03-19T21:18:17.432 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.430+0000 7f663dd9b700 1 --2- 172.21.15.170:0/3058361741 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f66400907c0 0x7f66401228a0 secure :-1 s=THROTTLE_DONE pgs=133 cs=0 l=1 rx=0x7f6640090c70 tx=0x7f6640067850).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:17.432 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.430+0000 7f662f7fe700 1 -- 172.21.15.170:0/3058361741 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+8837 (secure 0 0 0) 0x7f664011fdd0 con 0x7f66400907c0 2020-03-19T21:18:17.432 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:18:17.433 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":16,"fsid":"12d0a33b-6b40-4398-90b4-6b5df1da388e","created":"2020-03-19T21:17:59.135747+0000","modified":"2020-03-19T21:18:14.515226+0000","last_up_change":"2020-03-19T21:18:05.158515+0000","last_in_change":"2020-03-19T21:18:05.158515+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":3,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2020-03-19T21:18:08.391604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2020-03-19T21:18:11.286536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2020-03-19T21:18:12.597110+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"32ca3517-1077-4102-9aa4-edfcebe8e3d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":20680},{"type":"v1","addr":"172.21.15.170:6801","nonce":20680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":20680},{"type":"v1","addr":"172.21.15.170:6803","nonce":20680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":20680},{"type":"v1","addr":"172.21.15.170:6807","nonce":20680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":20680},{"type":"v1","addr":"172.21.15.170:6805","nonce":20680}]},"public_addr":"172.21.15.170:6801/20680","cluster_addr":"172.21.15.170:6803/20680","heartbeat_back_addr":"172.21.15.170:6807/20680","heartbeat_front_addr":"172.21.15.170:6805/20680","state":["exists","up"]},{"osd":1,"uuid":"470df64b-60bc-4f5b-9b53-64495cd7c5e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":20681},{"type":"v1","addr":"172.21.15.170:6809","nonce":20681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":20681},{"type":"v1","addr":"172.21.15.170:6811","nonce":20681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":20681},{"type":"v1","addr":"172.21.15.170:6815","nonce":20681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":20681},{"type":"v1","addr":"172.21.15.170:6813","nonce":20681}]},"public_addr":"172.21.15.170:6809/20681","cluster_addr":"172.21.15.170:6811/20681","heartbeat_back_addr":"172.21.15.170:6815/20681","heartbeat_front_addr":"172.21.15.170:6813/20681","state":["exists","up"]},{"osd":2,"uuid":"ccfb9d82-b759-4ff4-8ddf-4f1d741bac5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":20682},{"type":"v1","addr":"172.21.15.170:6817","nonce":20682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":20682},{"type":"v1","addr":"172.21.15.170:6819","nonce":20682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":20682},{"type":"v1","addr":"172.21.15.170:6823","nonce":20682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":20682},{"type":"v1","addr":"172.21.15.170:6821","nonce":20682}]},"public_addr":"172.21.15.170:6817/20682","cluster_addr":"172.21.15.170:6819/20682","heartbeat_back_addr":"172.21.15.170:6823/20682","heartbeat_front_addr":"172.21.15.170:6821/20682","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2020-03-19T21:18:03.177293+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2020-03-19T21:18:03.263179+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2020-03-19T21:18:03.183223+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-03-19T21:18:17.434 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.431+0000 7f6644991700 1 -- 172.21.15.170:0/3058361741 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f661c028760 msgr2=0x7f661c02ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:17.434 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.431+0000 7f6644991700 1 --2- 172.21.15.170:0/3058361741 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f661c028760 0x7f661c02ac20 secure :-1 s=READY pgs=33 cs=0 l=1 rx=0x7f6634005520 tx=0x7f66340016a0).stop 2020-03-19T21:18:17.434 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.431+0000 7f6644991700 1 -- 172.21.15.170:0/3058361741 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f66400907c0 msgr2=0x7f66401228a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:17.434 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.431+0000 7f6644991700 1 --2- 172.21.15.170:0/3058361741 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f66400907c0 0x7f66401228a0 secure :-1 s=READY pgs=133 cs=0 l=1 rx=0x7f6640090c70 tx=0x7f6640067850).stop 2020-03-19T21:18:17.435 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.431+0000 7f6644991700 1 -- 172.21.15.170:0/3058361741 shutdown_connections 2020-03-19T21:18:17.435 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.431+0000 7f6644991700 1 --2- 172.21.15.170:0/3058361741 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f661c028760 0x7f661c02ac20 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:17.435 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.431+0000 7f6644991700 1 --2- 172.21.15.170:0/3058361741 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f66400907c0 0x7f66401228a0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:17.435 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.431+0000 7f6644991700 1 -- 172.21.15.170:0/3058361741 >> 172.21.15.170:0/3058361741 conn(0x7f6640091540 msgr2=0x7f66400629b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:17.435 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.431+0000 7f6644991700 1 -- 172.21.15.170:0/3058361741 shutdown_connections 2020-03-19T21:18:17.435 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.431+0000 7f6644991700 1 -- 172.21.15.170:0/3058361741 wait complete. 2020-03-19T21:18:17.444 INFO:tasks.ceph.ceph_manager.ceph:all up! 2020-03-19T21:18:17.444 INFO:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-03-19T21:18:17.549 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.547+0000 7fa2d992a700 1 Processor -- start 2020-03-19T21:18:17.550 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.547+0000 7fa2d992a700 1 -- start start 2020-03-19T21:18:17.556 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.548+0000 7fa2d992a700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fa2d408d000 0x7fa2d40907c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T21:18:17.556 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.548+0000 7fa2d992a700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2d40605e0 con 0x7fa2d408d3e0 2020-03-19T21:18:17.556 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.548+0000 7fa2d992a700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2d408ef70 con 0x7fa2d408d000 2020-03-19T21:18:17.556 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.548+0000 7fa2d2ffd700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fa2d408d000 0x7fa2d40907c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:17.556 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.548+0000 7fa2d27fc700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fa2d408d3e0 0x7fa2d408d890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:53728/0 (socket says 172.21.15.170:53728) 2020-03-19T21:18:17.557 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.548+0000 7fa2d27fc700 1 -- 172.21.15.170:0/3095154929 learned_addr learned my addr 172.21.15.170:0/3095154929 (peer_addr_for_me v1:172.21.15.170:0/0) 2020-03-19T21:18:17.557 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.548+0000 7fa2d2ffd700 1 -- 172.21.15.170:0/3095154929 >> v1:172.21.15.170:6789/0 conn(0x7fa2d408d3e0 legacy=0x7fa2d408d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:17.557 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.548+0000 7fa2d1ffb700 1 -- 172.21.15.170:0/3095154929 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3983276214 0 0) 0x7fa2d40605e0 con 0x7fa2d408d3e0 2020-03-19T21:18:17.557 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.548+0000 7fa2d2ffd700 1 -- 172.21.15.170:0/3095154929 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2d4060810 con 0x7fa2d408d000 2020-03-19T21:18:17.557 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.548+0000 7fa2d2ffd700 1 --2- 172.21.15.170:0/3095154929 >> v2:172.21.15.170:3300/0 conn(0x7fa2d408d000 0x7fa2d40907c0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7fa2c8002f30 tx=0x7fa2c8007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:17.557 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.548+0000 7fa2d2ffd700 1 --2- 172.21.15.170:0/3095154929 >> v2:172.21.15.170:3300/0 conn(0x7fa2d408d000 0x7fa2d40907c0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7fa2c8002f30 tx=0x7fa2c8007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:17.557 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.548+0000 7fa2d2ffd700 1 --2- 172.21.15.170:0/3095154929 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa2d408d000 0x7fa2d40907c0 secure :-1 s=READY pgs=135 cs=0 l=1 rx=0x7fa2c8002f30 tx=0x7fa2c8007a40).ready entity=mon.0 client_cookie=8712f6f73fde1e7e server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:17.558 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.548+0000 7fa2d2ffd700 1 --2- 172.21.15.170:0/3095154929 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa2d408d000 0x7fa2d40907c0 secure :-1 s=THROTTLE_DONE pgs=135 cs=0 l=1 rx=0x7fa2c8002f30 tx=0x7fa2c8007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:17.558 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.548+0000 7fa2d1ffb700 1 -- 172.21.15.170:0/3095154929 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fa2c80113f0 con 0x7fa2d408d000 2020-03-19T21:18:17.558 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.548+0000 7fa2d2ffd700 1 --2- 172.21.15.170:0/3095154929 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa2d408d000 0x7fa2d40907c0 secure :-1 s=THROTTLE_DONE pgs=135 cs=0 l=1 rx=0x7fa2c8002f30 tx=0x7fa2c8007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:17.558 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.548+0000 7fa2d1ffb700 1 -- 172.21.15.170:0/3095154929 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2c8011550 con 0x7fa2d408d000 2020-03-19T21:18:17.558 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.548+0000 7fa2d2ffd700 1 --2- 172.21.15.170:0/3095154929 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa2d408d000 0x7fa2d40907c0 secure :-1 s=THROTTLE_DONE pgs=135 cs=0 l=1 rx=0x7fa2c8002f30 tx=0x7fa2c8007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:17.558 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.548+0000 7fa2d1ffb700 1 -- 172.21.15.170:0/3095154929 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fa2c80117f0 con 0x7fa2d408d000 2020-03-19T21:18:17.558 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.548+0000 7fa2d992a700 1 -- 172.21.15.170:0/3095154929 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa2d408d000 msgr2=0x7fa2d40907c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:17.559 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.548+0000 7fa2d992a700 1 --2- 172.21.15.170:0/3095154929 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa2d408d000 0x7fa2d40907c0 secure :-1 s=READY pgs=135 cs=0 l=1 rx=0x7fa2c8002f30 tx=0x7fa2c8007a40).stop 2020-03-19T21:18:17.559 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.549+0000 7fa2d992a700 1 -- 172.21.15.170:0/3095154929 shutdown_connections 2020-03-19T21:18:17.559 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.549+0000 7fa2d992a700 1 --2- 172.21.15.170:0/3095154929 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa2d408d000 0x7fa2d40907c0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:17.559 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.549+0000 7fa2d992a700 1 -- 172.21.15.170:0/3095154929 >> 172.21.15.170:0/3095154929 conn(0x7fa2d4091540 msgr2=0x7fa2d4091950 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:17.559 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.549+0000 7fa2d992a700 1 -- 172.21.15.170:0/3095154929 shutdown_connections 2020-03-19T21:18:17.559 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.549+0000 7fa2d992a700 1 -- 172.21.15.170:0/3095154929 wait complete. 2020-03-19T21:18:17.559 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.550+0000 7fa2d992a700 1 Processor -- start 2020-03-19T21:18:17.560 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.550+0000 7fa2d992a700 1 -- start start 2020-03-19T21:18:17.560 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.550+0000 7fa2d992a700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fa2d408d000 0x7fa2d4122560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:17.560 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.550+0000 7fa2d992a700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2d4121350 con 0x7fa2d408d3e0 2020-03-19T21:18:17.560 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.550+0000 7fa2d992a700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2d411f010 con 0x7fa2d408d000 2020-03-19T21:18:17.560 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.550+0000 7fa2d2ffd700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fa2d408d000 0x7fa2d4122560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:17.560 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.550+0000 7fa2d2ffd700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fa2d408d000 0x7fa2d4122560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:52968/0 (socket says 172.21.15.170:52968) 2020-03-19T21:18:17.560 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.550+0000 7fa2d27fc700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fa2d408d3e0 0x7fa2d41229c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:53734/0 (socket says 172.21.15.170:53734) 2020-03-19T21:18:17.560 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.550+0000 7fa2d2ffd700 1 -- 172.21.15.170:0/126554764 learned_addr learned my addr 172.21.15.170:0/126554764 (peer_addr_for_me v2:172.21.15.170:0/0) 2020-03-19T21:18:17.561 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.550+0000 7fa2d2ffd700 1 -- 172.21.15.170:0/126554764 >> v1:172.21.15.170:6789/0 conn(0x7fa2d408d3e0 legacy=0x7fa2d41229c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:17.561 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.550+0000 7fa2c3fff700 1 -- 172.21.15.170:0/126554764 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2042186823 0 0) 0x7fa2d4121350 con 0x7fa2d408d3e0 2020-03-19T21:18:17.561 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.550+0000 7fa2d2ffd700 1 -- 172.21.15.170:0/126554764 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2d4121040 con 0x7fa2d408d000 2020-03-19T21:18:17.561 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.550+0000 7fa2d2ffd700 1 --2- 172.21.15.170:0/126554764 >> v2:172.21.15.170:3300/0 conn(0x7fa2d408d000 0x7fa2d4122560 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7fa2c8002f30 tx=0x7fa2c8007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:17.561 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.550+0000 7fa2d2ffd700 1 --2- 172.21.15.170:0/126554764 >> v2:172.21.15.170:3300/0 conn(0x7fa2d408d000 0x7fa2d4122560 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7fa2c8002f30 tx=0x7fa2c8007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:17.561 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.550+0000 7fa2d2ffd700 1 --2- 172.21.15.170:0/126554764 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa2d408d000 0x7fa2d4122560 secure :-1 s=READY pgs=137 cs=0 l=1 rx=0x7fa2c8002f30 tx=0x7fa2c8007a40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:17.562 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.551+0000 7fa2d2ffd700 1 --2- 172.21.15.170:0/126554764 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa2d408d000 0x7fa2d4122560 secure :-1 s=THROTTLE_DONE pgs=137 cs=0 l=1 rx=0x7fa2c8002f30 tx=0x7fa2c8007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:17.562 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.551+0000 7fa2c3fff700 1 -- 172.21.15.170:0/126554764 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fa2c8007150 con 0x7fa2d408d000 2020-03-19T21:18:17.562 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.551+0000 7fa2d2ffd700 1 --2- 172.21.15.170:0/126554764 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa2d408d000 0x7fa2d4122560 secure :-1 s=THROTTLE_DONE pgs=137 cs=0 l=1 rx=0x7fa2c8002f30 tx=0x7fa2c8007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:17.563 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.551+0000 7fa2d992a700 1 -- 172.21.15.170:0/126554764 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2d411eb90 con 0x7fa2d408d000 2020-03-19T21:18:17.563 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.551+0000 7fa2c3fff700 1 -- 172.21.15.170:0/126554764 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2c80072b0 con 0x7fa2d408d000 2020-03-19T21:18:17.563 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.551+0000 7fa2d2ffd700 1 --2- 172.21.15.170:0/126554764 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa2d408d000 0x7fa2d4122560 secure :-1 s=THROTTLE_DONE pgs=137 cs=0 l=1 rx=0x7fa2c8002f30 tx=0x7fa2c8007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:17.563 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.551+0000 7fa2c3fff700 1 -- 172.21.15.170:0/126554764 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fa2c8007420 con 0x7fa2d408d000 2020-03-19T21:18:17.563 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.551+0000 7fa2d992a700 1 -- 172.21.15.170:0/126554764 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2d411fe20 con 0x7fa2d408d000 2020-03-19T21:18:17.563 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.551+0000 7fa2d2ffd700 1 --2- 172.21.15.170:0/126554764 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa2d408d000 0x7fa2d4122560 secure :-1 s=THROTTLE_DONE pgs=137 cs=0 l=1 rx=0x7fa2c8002f30 tx=0x7fa2c8007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:17.563 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.551+0000 7fa2c3fff700 1 -- 172.21.15.170:0/126554764 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 4) v1 ==== 29603+0+0 (secure 0 0 0) 0x7fa2c80056c0 con 0x7fa2d408d000 2020-03-19T21:18:17.564 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.551+0000 7fa2d2ffd700 1 --2- 172.21.15.170:0/126554764 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa2d408d000 0x7fa2d4122560 secure :-1 s=THROTTLE_DONE pgs=137 cs=0 l=1 rx=0x7fa2c8002f30 tx=0x7fa2c8007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:17.564 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.551+0000 7fa2c3fff700 1 --2- 172.21.15.170:0/126554764 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7fa2b0028200 0x7fa2b002a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:17.564 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.552+0000 7fa2c3fff700 1 -- 172.21.15.170:0/126554764 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3403+0+0 (secure 0 0 0) 0x7fa2c8034970 con 0x7fa2d408d000 2020-03-19T21:18:17.564 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.552+0000 7fa2d27fc700 1 --2- 172.21.15.170:0/126554764 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7fa2b0028200 0x7fa2b002a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:17.564 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.552+0000 7fa2d992a700 1 -- 172.21.15.170:0/126554764 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2d4056db0 con 0x7fa2d408d000 2020-03-19T21:18:17.564 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.552+0000 7fa2d27fc700 1 --2- 172.21.15.170:0/126554764 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7fa2b0028200 0x7fa2b002a6c0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7fa2c4000ee0 tx=0x7fa2c4001670).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:17.564 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.552+0000 7fa2d27fc700 1 --2- 172.21.15.170:0/126554764 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7fa2b0028200 0x7fa2b002a6c0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7fa2c4000ee0 tx=0x7fa2c4001670).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:17.565 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.552+0000 7fa2d27fc700 1 --2- 172.21.15.170:0/126554764 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7fa2b0028200 0x7fa2b002a6c0 secure :-1 s=READY pgs=34 cs=0 l=1 rx=0x7fa2c4000ee0 tx=0x7fa2c4001670).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:17.565 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.555+0000 7fa2d2ffd700 1 --2- 172.21.15.170:0/126554764 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa2d408d000 0x7fa2d4122560 secure :-1 s=THROTTLE_DONE pgs=137 cs=0 l=1 rx=0x7fa2c8002f30 tx=0x7fa2c8007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:17.565 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.555+0000 7fa2c3fff700 1 -- 172.21.15.170:0/126554764 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+126772 (secure 0 0 0) 0x7fa2c8006610 con 0x7fa2d408d000 2020-03-19T21:18:17.697 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.695+0000 7fa2d992a700 1 -- 172.21.15.170:0/126554764 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa2d406aaf0 con 0x7fa2d408d000 2020-03-19T21:18:17.698 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.696+0000 7fa2d2ffd700 1 --2- 172.21.15.170:0/126554764 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa2d408d000 0x7fa2d4122560 secure :-1 s=THROTTLE_DONE pgs=137 cs=0 l=1 rx=0x7fa2c8002f30 tx=0x7fa2c8007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:17.698 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.696+0000 7fa2c3fff700 1 -- 172.21.15.170:0/126554764 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+8837 (secure 0 0 0) 0x7fa2c8006070 con 0x7fa2d408d000 2020-03-19T21:18:17.698 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:18:17.699 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":16,"fsid":"12d0a33b-6b40-4398-90b4-6b5df1da388e","created":"2020-03-19T21:17:59.135747+0000","modified":"2020-03-19T21:18:14.515226+0000","last_up_change":"2020-03-19T21:18:05.158515+0000","last_in_change":"2020-03-19T21:18:05.158515+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":3,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2020-03-19T21:18:08.391604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2020-03-19T21:18:11.286536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2020-03-19T21:18:12.597110+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"32ca3517-1077-4102-9aa4-edfcebe8e3d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":20680},{"type":"v1","addr":"172.21.15.170:6801","nonce":20680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":20680},{"type":"v1","addr":"172.21.15.170:6803","nonce":20680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":20680},{"type":"v1","addr":"172.21.15.170:6807","nonce":20680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":20680},{"type":"v1","addr":"172.21.15.170:6805","nonce":20680}]},"public_addr":"172.21.15.170:6801/20680","cluster_addr":"172.21.15.170:6803/20680","heartbeat_back_addr":"172.21.15.170:6807/20680","heartbeat_front_addr":"172.21.15.170:6805/20680","state":["exists","up"]},{"osd":1,"uuid":"470df64b-60bc-4f5b-9b53-64495cd7c5e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":20681},{"type":"v1","addr":"172.21.15.170:6809","nonce":20681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":20681},{"type":"v1","addr":"172.21.15.170:6811","nonce":20681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":20681},{"type":"v1","addr":"172.21.15.170:6815","nonce":20681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":20681},{"type":"v1","addr":"172.21.15.170:6813","nonce":20681}]},"public_addr":"172.21.15.170:6809/20681","cluster_addr":"172.21.15.170:6811/20681","heartbeat_back_addr":"172.21.15.170:6815/20681","heartbeat_front_addr":"172.21.15.170:6813/20681","state":["exists","up"]},{"osd":2,"uuid":"ccfb9d82-b759-4ff4-8ddf-4f1d741bac5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":20682},{"type":"v1","addr":"172.21.15.170:6817","nonce":20682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":20682},{"type":"v1","addr":"172.21.15.170:6819","nonce":20682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":20682},{"type":"v1","addr":"172.21.15.170:6823","nonce":20682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":20682},{"type":"v1","addr":"172.21.15.170:6821","nonce":20682}]},"public_addr":"172.21.15.170:6817/20682","cluster_addr":"172.21.15.170:6819/20682","heartbeat_back_addr":"172.21.15.170:6823/20682","heartbeat_front_addr":"172.21.15.170:6821/20682","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2020-03-19T21:18:03.177293+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2020-03-19T21:18:03.263179+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2020-03-19T21:18:03.183223+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-03-19T21:18:17.700 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.697+0000 7fa2d992a700 1 -- 172.21.15.170:0/126554764 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7fa2b0028200 msgr2=0x7fa2b002a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:17.700 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.697+0000 7fa2d992a700 1 --2- 172.21.15.170:0/126554764 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7fa2b0028200 0x7fa2b002a6c0 secure :-1 s=READY pgs=34 cs=0 l=1 rx=0x7fa2c4000ee0 tx=0x7fa2c4001670).stop 2020-03-19T21:18:17.700 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.697+0000 7fa2d992a700 1 -- 172.21.15.170:0/126554764 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa2d408d000 msgr2=0x7fa2d4122560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:17.701 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.697+0000 7fa2d992a700 1 --2- 172.21.15.170:0/126554764 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa2d408d000 0x7fa2d4122560 secure :-1 s=READY pgs=137 cs=0 l=1 rx=0x7fa2c8002f30 tx=0x7fa2c8007a40).stop 2020-03-19T21:18:17.701 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.697+0000 7fa2d992a700 1 -- 172.21.15.170:0/126554764 shutdown_connections 2020-03-19T21:18:17.701 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.697+0000 7fa2d992a700 1 --2- 172.21.15.170:0/126554764 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7fa2b0028200 0x7fa2b002a6c0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:17.701 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.697+0000 7fa2d992a700 1 --2- 172.21.15.170:0/126554764 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa2d408d000 0x7fa2d4122560 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:17.701 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.697+0000 7fa2d992a700 1 -- 172.21.15.170:0/126554764 >> 172.21.15.170:0/126554764 conn(0x7fa2d4091540 msgr2=0x7fa2d40629b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:17.702 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.697+0000 7fa2d992a700 1 -- 172.21.15.170:0/126554764 shutdown_connections 2020-03-19T21:18:17.702 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.697+0000 7fa2d992a700 1 -- 172.21.15.170:0/126554764 wait complete. 2020-03-19T21:18:17.710 INFO:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early tell osd.0 flush_pg_stats 2020-03-19T21:18:17.816 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.814+0000 7fe1bcbf7700 1 Processor -- start 2020-03-19T21:18:17.816 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.814+0000 7fe1bcbf7700 1 -- start start 2020-03-19T21:18:17.821 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.814+0000 7fe1bcbf7700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fe1b811fd60 0x7fe1b8127a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T21:18:17.822 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.814+0000 7fe1bcbf7700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1b80911c0 con 0x7fe1b8091eb0 2020-03-19T21:18:17.822 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.814+0000 7fe1bcbf7700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1b8067130 con 0x7fe1b811fd60 2020-03-19T21:18:17.822 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.815+0000 7fe1b5d9b700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fe1b811fd60 0x7fe1b8127a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:17.822 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.815+0000 7fe1b659c700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fe1b8091eb0 0x7fe1b811f650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:53738/0 (socket says 172.21.15.170:53738) 2020-03-19T21:18:17.822 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.815+0000 7fe1b659c700 1 -- 172.21.15.170:0/1335290310 learned_addr learned my addr 172.21.15.170:0/1335290310 (peer_addr_for_me v1:172.21.15.170:0/0) 2020-03-19T21:18:17.822 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.815+0000 7fe1b5d9b700 1 --2- 172.21.15.170:0/1335290310 >> v2:172.21.15.170:3300/0 conn(0x7fe1b811fd60 0x7fe1b8127a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:52974/0 (socket says 172.21.15.170:52974) 2020-03-19T21:18:17.823 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.815+0000 7fe1b5d9b700 1 -- 172.21.15.170:0/1335290310 >> v1:172.21.15.170:6789/0 conn(0x7fe1b8091eb0 legacy=0x7fe1b811f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:17.823 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.815+0000 7fe1b5d9b700 1 -- 172.21.15.170:0/1335290310 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1b811f340 con 0x7fe1b811fd60 2020-03-19T21:18:17.823 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.815+0000 7fe1b559a700 1 -- 172.21.15.170:0/1335290310 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 61295959 0 0) 0x7fe1b80911c0 con 0x7fe1b8091eb0 2020-03-19T21:18:17.823 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.815+0000 7fe1b5d9b700 1 --2- 172.21.15.170:0/1335290310 >> v2:172.21.15.170:3300/0 conn(0x7fe1b811fd60 0x7fe1b8127a70 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7fe1a0002f30 tx=0x7fe1a0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:17.823 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.815+0000 7fe1b5d9b700 1 --2- 172.21.15.170:0/1335290310 >> v2:172.21.15.170:3300/0 conn(0x7fe1b811fd60 0x7fe1b8127a70 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7fe1a0002f30 tx=0x7fe1a0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:17.823 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.815+0000 7fe1b5d9b700 1 --2- 172.21.15.170:0/1335290310 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe1b811fd60 0x7fe1b8127a70 secure :-1 s=READY pgs=139 cs=0 l=1 rx=0x7fe1a0002f30 tx=0x7fe1a0007a40).ready entity=mon.0 client_cookie=d311d443a2c94f01 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:17.823 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.815+0000 7fe1b5d9b700 1 --2- 172.21.15.170:0/1335290310 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe1b811fd60 0x7fe1b8127a70 secure :-1 s=THROTTLE_DONE pgs=139 cs=0 l=1 rx=0x7fe1a0002f30 tx=0x7fe1a0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:17.824 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.815+0000 7fe1b559a700 1 -- 172.21.15.170:0/1335290310 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fe1a0008e80 con 0x7fe1b811fd60 2020-03-19T21:18:17.824 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.815+0000 7fe1b5d9b700 1 --2- 172.21.15.170:0/1335290310 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe1b811fd60 0x7fe1b8127a70 secure :-1 s=THROTTLE_DONE pgs=139 cs=0 l=1 rx=0x7fe1a0002f30 tx=0x7fe1a0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:17.824 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.815+0000 7fe1b559a700 1 -- 172.21.15.170:0/1335290310 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1a00113f0 con 0x7fe1b811fd60 2020-03-19T21:18:17.824 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.815+0000 7fe1b5d9b700 1 --2- 172.21.15.170:0/1335290310 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe1b811fd60 0x7fe1b8127a70 secure :-1 s=THROTTLE_DONE pgs=139 cs=0 l=1 rx=0x7fe1a0002f30 tx=0x7fe1a0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:17.824 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.815+0000 7fe1b559a700 1 -- 172.21.15.170:0/1335290310 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fe1a0011690 con 0x7fe1b811fd60 2020-03-19T21:18:17.824 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.815+0000 7fe1bcbf7700 1 -- 172.21.15.170:0/1335290310 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe1b811fd60 msgr2=0x7fe1b8127a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:17.824 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.815+0000 7fe1bcbf7700 1 --2- 172.21.15.170:0/1335290310 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe1b811fd60 0x7fe1b8127a70 secure :-1 s=READY pgs=139 cs=0 l=1 rx=0x7fe1a0002f30 tx=0x7fe1a0007a40).stop 2020-03-19T21:18:17.824 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.816+0000 7fe1bcbf7700 1 -- 172.21.15.170:0/1335290310 shutdown_connections 2020-03-19T21:18:17.825 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.816+0000 7fe1bcbf7700 1 --2- 172.21.15.170:0/1335290310 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe1b811fd60 0x7fe1b8127a70 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:17.825 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.816+0000 7fe1bcbf7700 1 -- 172.21.15.170:0/1335290310 >> 172.21.15.170:0/1335290310 conn(0x7fe1b8091390 msgr2=0x7fe1b80917a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:17.825 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.816+0000 7fe1bcbf7700 1 -- 172.21.15.170:0/1335290310 shutdown_connections 2020-03-19T21:18:17.825 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.816+0000 7fe1bcbf7700 1 -- 172.21.15.170:0/1335290310 wait complete. 2020-03-19T21:18:17.825 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.816+0000 7fe1bcbf7700 1 Processor -- start 2020-03-19T21:18:17.825 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.816+0000 7fe1bcbf7700 1 -- start start 2020-03-19T21:18:17.825 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.817+0000 7fe1bcbf7700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fe1b811fd60 0x7fe1b8063e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:17.826 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.817+0000 7fe1bcbf7700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1b8066bd0 con 0x7fe1b8091eb0 2020-03-19T21:18:17.826 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.817+0000 7fe1bcbf7700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1b8066430 con 0x7fe1b811fd60 2020-03-19T21:18:17.826 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.817+0000 7fe1b659c700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fe1b8091eb0 0x7fe1b8063740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:53742/0 (socket says 172.21.15.170:53742) 2020-03-19T21:18:17.826 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.817+0000 7fe1b659c700 1 -- 172.21.15.170:0/811370586 learned_addr learned my addr 172.21.15.170:0/811370586 (peer_addr_for_me v1:172.21.15.170:0/0) 2020-03-19T21:18:17.826 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.817+0000 7fe1b5d9b700 1 --2- 172.21.15.170:0/811370586 >> v2:172.21.15.170:3300/0 conn(0x7fe1b811fd60 0x7fe1b8063e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:17.826 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.817+0000 7fe1b5d9b700 1 -- 172.21.15.170:0/811370586 >> v1:172.21.15.170:6789/0 conn(0x7fe1b8091eb0 legacy=0x7fe1b8063740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:17.826 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.817+0000 7fe1af7fe700 1 -- 172.21.15.170:0/811370586 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1561459318 0 0) 0x7fe1b8066bd0 con 0x7fe1b8091eb0 2020-03-19T21:18:17.827 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.817+0000 7fe1b5d9b700 1 -- 172.21.15.170:0/811370586 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1b8065540 con 0x7fe1b811fd60 2020-03-19T21:18:17.827 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.817+0000 7fe1b5d9b700 1 --2- 172.21.15.170:0/811370586 >> v2:172.21.15.170:3300/0 conn(0x7fe1b811fd60 0x7fe1b8063e50 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7fe1a0002f30 tx=0x7fe1a0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:17.827 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.817+0000 7fe1b5d9b700 1 --2- 172.21.15.170:0/811370586 >> v2:172.21.15.170:3300/0 conn(0x7fe1b811fd60 0x7fe1b8063e50 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7fe1a0002f30 tx=0x7fe1a0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:17.827 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.817+0000 7fe1b5d9b700 1 --2- 172.21.15.170:0/811370586 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe1b811fd60 0x7fe1b8063e50 secure :-1 s=READY pgs=141 cs=0 l=1 rx=0x7fe1a0002f30 tx=0x7fe1a0007a40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:17.827 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.817+0000 7fe1b5d9b700 1 --2- 172.21.15.170:0/811370586 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe1b811fd60 0x7fe1b8063e50 secure :-1 s=THROTTLE_DONE pgs=141 cs=0 l=1 rx=0x7fe1a0002f30 tx=0x7fe1a0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:17.827 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.817+0000 7fe1af7fe700 1 -- 172.21.15.170:0/811370586 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fe1a0011e90 con 0x7fe1b811fd60 2020-03-19T21:18:17.827 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.817+0000 7fe1b5d9b700 1 --2- 172.21.15.170:0/811370586 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe1b811fd60 0x7fe1b8063e50 secure :-1 s=THROTTLE_DONE pgs=141 cs=0 l=1 rx=0x7fe1a0002f30 tx=0x7fe1a0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:17.828 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.817+0000 7fe1b5d9b700 1 --2- 172.21.15.170:0/811370586 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe1b811fd60 0x7fe1b8063e50 secure :-1 s=THROTTLE_DONE pgs=141 cs=0 l=1 rx=0x7fe1a0002f30 tx=0x7fe1a0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:17.828 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.817+0000 7fe1af7fe700 1 -- 172.21.15.170:0/811370586 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1a000f830 con 0x7fe1b811fd60 2020-03-19T21:18:17.828 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.817+0000 7fe1bcbf7700 1 -- 172.21.15.170:0/811370586 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1b811cc90 con 0x7fe1b811fd60 2020-03-19T21:18:17.828 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.817+0000 7fe1af7fe700 1 -- 172.21.15.170:0/811370586 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fe1a0007280 con 0x7fe1b811fd60 2020-03-19T21:18:17.828 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.817+0000 7fe1bcbf7700 1 -- 172.21.15.170:0/811370586 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1b8065120 con 0x7fe1b811fd60 2020-03-19T21:18:17.829 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.818+0000 7fe1b5d9b700 1 --2- 172.21.15.170:0/811370586 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe1b811fd60 0x7fe1b8063e50 secure :-1 s=THROTTLE_DONE pgs=141 cs=0 l=1 rx=0x7fe1a0002f30 tx=0x7fe1a0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:17.829 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.818+0000 7fe1af7fe700 1 -- 172.21.15.170:0/811370586 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 4) v1 ==== 29603+0+0 (secure 0 0 0) 0x7fe1a00074b0 con 0x7fe1b811fd60 2020-03-19T21:18:17.829 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.818+0000 7fe1b5d9b700 1 --2- 172.21.15.170:0/811370586 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe1b811fd60 0x7fe1b8063e50 secure :-1 s=THROTTLE_DONE pgs=141 cs=0 l=1 rx=0x7fe1a0002f30 tx=0x7fe1a0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:17.829 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.818+0000 7fe1af7fe700 1 --2- 172.21.15.170:0/811370586 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7fe19c028280 0x7fe19c02a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:17.829 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.818+0000 7fe1af7fe700 1 -- 172.21.15.170:0/811370586 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3403+0+0 (secure 0 0 0) 0x7fe1a0034930 con 0x7fe1b811fd60 2020-03-19T21:18:17.830 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.818+0000 7fe1b659c700 1 --2- 172.21.15.170:0/811370586 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7fe19c028280 0x7fe19c02a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:17.830 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.818+0000 7fe1b659c700 1 --2- 172.21.15.170:0/811370586 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7fe19c028280 0x7fe19c02a740 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7fe1b00015e0 tx=0x7fe1b0001aa0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:17.830 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.818+0000 7fe1bcbf7700 1 --2- 172.21.15.170:0/811370586 >> [v2:172.21.15.170:6800/20680,v1:172.21.15.170:6801/20680] conn(0x7fe1b8055b30 0x7fe1b8055f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:17.830 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.818+0000 7fe1bcbf7700 1 -- 172.21.15.170:0/811370586 --> [v2:172.21.15.170:6800/20680,v1:172.21.15.170:6801/20680] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe1b8065740 con 0x7fe1b8055b30 2020-03-19T21:18:17.830 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.818+0000 7fe1b659c700 1 --2- 172.21.15.170:0/811370586 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7fe19c028280 0x7fe19c02a740 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7fe1b00015e0 tx=0x7fe1b0001aa0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:17.831 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.818+0000 7fe1b659c700 1 --2- 172.21.15.170:0/811370586 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7fe19c028280 0x7fe19c02a740 secure :-1 s=READY pgs=35 cs=0 l=1 rx=0x7fe1b00015e0 tx=0x7fe1b0001aa0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:17.831 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.819+0000 7fe1b6d9d700 1 --2- 172.21.15.170:0/811370586 >> [v2:172.21.15.170:6800/20680,v1:172.21.15.170:6801/20680] conn(0x7fe1b8055b30 0x7fe1b8055f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:17.831 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.819+0000 7fe1b6d9d700 1 --2- 172.21.15.170:0/811370586 >> [v2:172.21.15.170:6800/20680,v1:172.21.15.170:6801/20680] conn(0x7fe1b8055b30 0x7fe1b8055f10 crc :-1 s=READY pgs=6 cs=0 l=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:17.831 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.820+0000 7fe1af7fe700 1 -- 172.21.15.170:0/811370586 <== osd.0 v2:172.21.15.170:6800/20680 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23288 (crc 0 0 0) 0x7fe1a8008250 con 0x7fe1b8055b30 2020-03-19T21:18:17.839 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.837+0000 7fe1bcbf7700 1 -- 172.21.15.170:0/811370586 --> [v2:172.21.15.170:6800/20680,v1:172.21.15.170:6801/20680] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe1b80693f0 con 0x7fe1b8055b30 2020-03-19T21:18:17.841 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.837+0000 7fe1af7fe700 1 -- 172.21.15.170:0/811370586 <== osd.0 v2:172.21.15.170:6800/20680 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fe1b80693f0 con 0x7fe1b8055b30 2020-03-19T21:18:17.841 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.838+0000 7fe1bcbf7700 1 -- 172.21.15.170:0/811370586 >> [v2:172.21.15.170:6800/20680,v1:172.21.15.170:6801/20680] conn(0x7fe1b8055b30 msgr2=0x7fe1b8055f10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:17.841 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.838+0000 7fe1bcbf7700 1 --2- 172.21.15.170:0/811370586 >> [v2:172.21.15.170:6800/20680,v1:172.21.15.170:6801/20680] conn(0x7fe1b8055b30 0x7fe1b8055f10 crc :-1 s=READY pgs=6 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:17.841 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.838+0000 7fe1bcbf7700 1 -- 172.21.15.170:0/811370586 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7fe19c028280 msgr2=0x7fe19c02a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:17.842 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.838+0000 7fe1bcbf7700 1 --2- 172.21.15.170:0/811370586 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7fe19c028280 0x7fe19c02a740 secure :-1 s=READY pgs=35 cs=0 l=1 rx=0x7fe1b00015e0 tx=0x7fe1b0001aa0).stop 2020-03-19T21:18:17.842 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.838+0000 7fe1bcbf7700 1 -- 172.21.15.170:0/811370586 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe1b811fd60 msgr2=0x7fe1b8063e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:17.842 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.838+0000 7fe1bcbf7700 1 --2- 172.21.15.170:0/811370586 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe1b811fd60 0x7fe1b8063e50 secure :-1 s=READY pgs=141 cs=0 l=1 rx=0x7fe1a0002f30 tx=0x7fe1a0007a40).stop 2020-03-19T21:18:17.842 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.838+0000 7fe1bcbf7700 1 -- 172.21.15.170:0/811370586 shutdown_connections 2020-03-19T21:18:17.842 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.838+0000 7fe1bcbf7700 1 --2- 172.21.15.170:0/811370586 >> [v2:172.21.15.170:6800/20680,v1:172.21.15.170:6801/20680] conn(0x7fe1b8055b30 0x7fe1b8055f10 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:17.843 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.838+0000 7fe1bcbf7700 1 --2- 172.21.15.170:0/811370586 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7fe19c028280 0x7fe19c02a740 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:17.843 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.838+0000 7fe1bcbf7700 1 --2- 172.21.15.170:0/811370586 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe1b811fd60 0x7fe1b8063e50 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:17.843 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.838+0000 7fe1bcbf7700 1 -- 172.21.15.170:0/811370586 >> 172.21.15.170:0/811370586 conn(0x7fe1b8091390 msgr2=0x7fe1b8060110 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:17.843 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.838+0000 7fe1bcbf7700 1 -- 172.21.15.170:0/811370586 shutdown_connections 2020-03-19T21:18:17.843 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.838+0000 7fe1bcbf7700 1 -- 172.21.15.170:0/811370586 wait complete. 2020-03-19T21:18:17.849 INFO:teuthology.orchestra.run.smithi170.stdout:34359738373 2020-03-19T21:18:17.850 INFO:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early tell osd.1 flush_pg_stats 2020-03-19T21:18:17.965 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.953+0000 7f108e2a0700 1 Processor -- start 2020-03-19T21:18:17.965 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.953+0000 7f108e2a0700 1 -- start start 2020-03-19T21:18:17.966 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.954+0000 7f108e2a0700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f108811fd60 0x7f1088127a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T21:18:17.966 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.954+0000 7f108e2a0700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10880911c0 con 0x7f1088091eb0 2020-03-19T21:18:17.966 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.954+0000 7f108e2a0700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1088067130 con 0x7f108811fd60 2020-03-19T21:18:17.967 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.954+0000 7f10877fe700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f108811fd60 0x7f1088127a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:17.967 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.954+0000 7f10877fe700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f108811fd60 0x7f1088127a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:52986/0 (socket says 172.21.15.170:52986) 2020-03-19T21:18:17.967 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.954+0000 7f1087fff700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f1088091eb0 0x7f108811f650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:53750/0 (socket says 172.21.15.170:53750) 2020-03-19T21:18:17.968 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.954+0000 7f10877fe700 1 -- 172.21.15.170:0/386288027 learned_addr learned my addr 172.21.15.170:0/386288027 (peer_addr_for_me v2:172.21.15.170:0/0) 2020-03-19T21:18:17.968 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.954+0000 7f10877fe700 1 -- 172.21.15.170:0/386288027 >> v1:172.21.15.170:6789/0 conn(0x7f1088091eb0 legacy=0x7f108811f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:17.968 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.954+0000 7f10877fe700 1 -- 172.21.15.170:0/386288027 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f108811f340 con 0x7f108811fd60 2020-03-19T21:18:17.970 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.954+0000 7f10877fe700 1 --2- 172.21.15.170:0/386288027 >> v2:172.21.15.170:3300/0 conn(0x7f108811fd60 0x7f1088127a70 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f107c002f30 tx=0x7f107c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:17.970 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.955+0000 7f10877fe700 1 --2- 172.21.15.170:0/386288027 >> v2:172.21.15.170:3300/0 conn(0x7f108811fd60 0x7f1088127a70 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f107c002f30 tx=0x7f107c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:17.970 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.955+0000 7f10877fe700 1 --2- 172.21.15.170:0/386288027 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f108811fd60 0x7f1088127a70 secure :-1 s=READY pgs=143 cs=0 l=1 rx=0x7f107c002f30 tx=0x7f107c007a40).ready entity=mon.0 client_cookie=dbdf756ae7b123f4 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:17.971 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.955+0000 7f10877fe700 1 --2- 172.21.15.170:0/386288027 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f108811fd60 0x7f1088127a70 secure :-1 s=THROTTLE_DONE pgs=143 cs=0 l=1 rx=0x7f107c002f30 tx=0x7f107c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:17.971 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.955+0000 7f1086ffd700 1 -- 172.21.15.170:0/386288027 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f107c0114b0 con 0x7f108811fd60 2020-03-19T21:18:17.971 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.955+0000 7f10877fe700 1 --2- 172.21.15.170:0/386288027 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f108811fd60 0x7f1088127a70 secure :-1 s=THROTTLE_DONE pgs=143 cs=0 l=1 rx=0x7f107c002f30 tx=0x7f107c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:17.972 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.955+0000 7f10877fe700 1 --2- 172.21.15.170:0/386288027 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f108811fd60 0x7f1088127a70 secure :-1 s=THROTTLE_DONE pgs=143 cs=0 l=1 rx=0x7f107c002f30 tx=0x7f107c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:17.972 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.955+0000 7f1086ffd700 1 -- 172.21.15.170:0/386288027 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f107c011610 con 0x7f108811fd60 2020-03-19T21:18:17.972 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.955+0000 7f1086ffd700 1 -- 172.21.15.170:0/386288027 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f107c011930 con 0x7f108811fd60 2020-03-19T21:18:17.973 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.955+0000 7f108e2a0700 1 -- 172.21.15.170:0/386288027 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f108811fd60 msgr2=0x7f1088127a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:17.973 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.955+0000 7f108e2a0700 1 --2- 172.21.15.170:0/386288027 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f108811fd60 0x7f1088127a70 secure :-1 s=READY pgs=143 cs=0 l=1 rx=0x7f107c002f30 tx=0x7f107c007a40).stop 2020-03-19T21:18:17.973 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.955+0000 7f108e2a0700 1 -- 172.21.15.170:0/386288027 shutdown_connections 2020-03-19T21:18:17.974 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.955+0000 7f108e2a0700 1 --2- 172.21.15.170:0/386288027 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f108811fd60 0x7f1088127a70 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:17.974 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.955+0000 7f108e2a0700 1 -- 172.21.15.170:0/386288027 >> 172.21.15.170:0/386288027 conn(0x7f1088091390 msgr2=0x7f10880917a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:17.975 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.955+0000 7f108e2a0700 1 -- 172.21.15.170:0/386288027 shutdown_connections 2020-03-19T21:18:17.975 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.955+0000 7f108e2a0700 1 -- 172.21.15.170:0/386288027 wait complete. 2020-03-19T21:18:17.975 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.956+0000 7f108e2a0700 1 Processor -- start 2020-03-19T21:18:17.976 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.956+0000 7f108e2a0700 1 -- start start 2020-03-19T21:18:17.976 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.956+0000 7f108e2a0700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f108811fd60 0x7f1088133bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:17.976 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.956+0000 7f108e2a0700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1088065040 con 0x7f1088091eb0 2020-03-19T21:18:17.976 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.956+0000 7f108e2a0700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f108810b6c0 con 0x7f108811fd60 2020-03-19T21:18:17.977 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.957+0000 7f10877fe700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f108811fd60 0x7f1088133bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:17.977 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.957+0000 7f10877fe700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f108811fd60 0x7f1088133bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:52992/0 (socket says 172.21.15.170:52992) 2020-03-19T21:18:17.977 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.957+0000 7f10877fe700 1 -- 172.21.15.170:0/2745664170 learned_addr learned my addr 172.21.15.170:0/2745664170 (peer_addr_for_me v2:172.21.15.170:0/0) 2020-03-19T21:18:17.978 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.957+0000 7f1087fff700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f1088091eb0 0x7f108812ee00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:53754/0 (socket says 172.21.15.170:53754) 2020-03-19T21:18:17.978 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.957+0000 7f10877fe700 1 -- 172.21.15.170:0/2745664170 >> v1:172.21.15.170:6789/0 conn(0x7f1088091eb0 legacy=0x7f108812ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:17.978 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.957+0000 7f10857fa700 1 -- 172.21.15.170:0/2745664170 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3515718039 0 0) 0x7f1088065040 con 0x7f1088091eb0 2020-03-19T21:18:17.978 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.957+0000 7f10877fe700 1 -- 172.21.15.170:0/2745664170 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f108811ce20 con 0x7f108811fd60 2020-03-19T21:18:17.979 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.957+0000 7f10877fe700 1 --2- 172.21.15.170:0/2745664170 >> v2:172.21.15.170:3300/0 conn(0x7f108811fd60 0x7f1088133bd0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f107c002f30 tx=0x7f107c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:17.979 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.957+0000 7f10877fe700 1 --2- 172.21.15.170:0/2745664170 >> v2:172.21.15.170:3300/0 conn(0x7f108811fd60 0x7f1088133bd0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f107c002f30 tx=0x7f107c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:17.979 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.957+0000 7f10877fe700 1 --2- 172.21.15.170:0/2745664170 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f108811fd60 0x7f1088133bd0 secure :-1 s=READY pgs=145 cs=0 l=1 rx=0x7f107c002f30 tx=0x7f107c007a40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:17.980 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.957+0000 7f10877fe700 1 --2- 172.21.15.170:0/2745664170 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f108811fd60 0x7f1088133bd0 secure :-1 s=THROTTLE_DONE pgs=145 cs=0 l=1 rx=0x7f107c002f30 tx=0x7f107c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:17.980 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.957+0000 7f10857fa700 1 -- 172.21.15.170:0/2745664170 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f107c00f8d0 con 0x7f108811fd60 2020-03-19T21:18:17.981 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.957+0000 7f10877fe700 1 --2- 172.21.15.170:0/2745664170 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f108811fd60 0x7f1088133bd0 secure :-1 s=THROTTLE_DONE pgs=145 cs=0 l=1 rx=0x7f107c002f30 tx=0x7f107c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:17.981 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.957+0000 7f10877fe700 1 --2- 172.21.15.170:0/2745664170 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f108811fd60 0x7f1088133bd0 secure :-1 s=THROTTLE_DONE pgs=145 cs=0 l=1 rx=0x7f107c002f30 tx=0x7f107c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:17.982 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.957+0000 7f10857fa700 1 -- 172.21.15.170:0/2745664170 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f107c007150 con 0x7f108811fd60 2020-03-19T21:18:17.982 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.957+0000 7f108e2a0700 1 -- 172.21.15.170:0/2745664170 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1088123b50 con 0x7f108811fd60 2020-03-19T21:18:17.982 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.957+0000 7f10857fa700 1 -- 172.21.15.170:0/2745664170 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f107c007470 con 0x7f108811fd60 2020-03-19T21:18:17.982 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.957+0000 7f108e2a0700 1 -- 172.21.15.170:0/2745664170 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10880657f0 con 0x7f108811fd60 2020-03-19T21:18:17.983 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.957+0000 7f10877fe700 1 --2- 172.21.15.170:0/2745664170 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f108811fd60 0x7f1088133bd0 secure :-1 s=THROTTLE_DONE pgs=145 cs=0 l=1 rx=0x7f107c002f30 tx=0x7f107c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:17.983 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.958+0000 7f10857fa700 1 -- 172.21.15.170:0/2745664170 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 4) v1 ==== 29603+0+0 (secure 0 0 0) 0x7f107c0056c0 con 0x7f108811fd60 2020-03-19T21:18:17.984 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.958+0000 7f10877fe700 1 --2- 172.21.15.170:0/2745664170 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f108811fd60 0x7f1088133bd0 secure :-1 s=THROTTLE_DONE pgs=145 cs=0 l=1 rx=0x7f107c002f30 tx=0x7f107c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:17.984 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.958+0000 7f10857fa700 1 --2- 172.21.15.170:0/2745664170 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f1068028260 0x7f106802a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:17.985 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.958+0000 7f10857fa700 1 -- 172.21.15.170:0/2745664170 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3403+0+0 (secure 0 0 0) 0x7f107c034950 con 0x7f108811fd60 2020-03-19T21:18:17.985 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.958+0000 7f1087fff700 1 --2- 172.21.15.170:0/2745664170 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f1068028260 0x7f106802a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:17.985 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.958+0000 7f108e2a0700 1 --2- 172.21.15.170:0/2745664170 >> [v2:172.21.15.170:6808/20681,v1:172.21.15.170:6809/20681] conn(0x7f1088056040 0x7f1088056470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:17.985 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.958+0000 7f108e2a0700 1 -- 172.21.15.170:0/2745664170 --> [v2:172.21.15.170:6808/20681,v1:172.21.15.170:6809/20681] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1088056db0 con 0x7f1088056040 2020-03-19T21:18:17.986 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.958+0000 7f1087fff700 1 --2- 172.21.15.170:0/2745664170 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f1068028260 0x7f106802a720 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f1078001160 tx=0x7f10780016c0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:17.987 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.958+0000 7f1087fff700 1 --2- 172.21.15.170:0/2745664170 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f1068028260 0x7f106802a720 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f1078001160 tx=0x7f10780016c0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:17.987 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.958+0000 7f1087fff700 1 --2- 172.21.15.170:0/2745664170 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f1068028260 0x7f106802a720 secure :-1 s=READY pgs=36 cs=0 l=1 rx=0x7f1078001160 tx=0x7f10780016c0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:17.987 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.958+0000 7f108c83d700 1 --2- 172.21.15.170:0/2745664170 >> [v2:172.21.15.170:6808/20681,v1:172.21.15.170:6809/20681] conn(0x7f1088056040 0x7f1088056470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:17.987 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.959+0000 7f108c83d700 1 --2- 172.21.15.170:0/2745664170 >> [v2:172.21.15.170:6808/20681,v1:172.21.15.170:6809/20681] conn(0x7f1088056040 0x7f1088056470 crc :-1 s=READY pgs=7 cs=0 l=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:17.987 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.959+0000 7f10857fa700 1 -- 172.21.15.170:0/2745664170 <== osd.1 v2:172.21.15.170:6808/20681 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23288 (crc 0 0 0) 0x7f1088056db0 con 0x7f1088056040 2020-03-19T21:18:17.989 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.976+0000 7f108e2a0700 1 -- 172.21.15.170:0/2745664170 --> [v2:172.21.15.170:6808/20681,v1:172.21.15.170:6809/20681] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1088056b40 con 0x7f1088056040 2020-03-19T21:18:17.992 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.977+0000 7f10857fa700 1 -- 172.21.15.170:0/2745664170 <== osd.1 v2:172.21.15.170:6808/20681 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f1088056b40 con 0x7f1088056040 2020-03-19T21:18:17.992 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.977+0000 7f108e2a0700 1 -- 172.21.15.170:0/2745664170 >> [v2:172.21.15.170:6808/20681,v1:172.21.15.170:6809/20681] conn(0x7f1088056040 msgr2=0x7f1088056470 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:17.992 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.977+0000 7f108e2a0700 1 --2- 172.21.15.170:0/2745664170 >> [v2:172.21.15.170:6808/20681,v1:172.21.15.170:6809/20681] conn(0x7f1088056040 0x7f1088056470 crc :-1 s=READY pgs=7 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:17.993 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.977+0000 7f108e2a0700 1 -- 172.21.15.170:0/2745664170 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f1068028260 msgr2=0x7f106802a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:17.993 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.977+0000 7f108e2a0700 1 --2- 172.21.15.170:0/2745664170 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f1068028260 0x7f106802a720 secure :-1 s=READY pgs=36 cs=0 l=1 rx=0x7f1078001160 tx=0x7f10780016c0).stop 2020-03-19T21:18:17.993 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.977+0000 7f108e2a0700 1 -- 172.21.15.170:0/2745664170 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f108811fd60 msgr2=0x7f1088133bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:17.994 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.977+0000 7f108e2a0700 1 --2- 172.21.15.170:0/2745664170 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f108811fd60 0x7f1088133bd0 secure :-1 s=READY pgs=145 cs=0 l=1 rx=0x7f107c002f30 tx=0x7f107c007a40).stop 2020-03-19T21:18:17.994 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.977+0000 7f108e2a0700 1 -- 172.21.15.170:0/2745664170 shutdown_connections 2020-03-19T21:18:17.994 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.977+0000 7f108e2a0700 1 --2- 172.21.15.170:0/2745664170 >> [v2:172.21.15.170:6808/20681,v1:172.21.15.170:6809/20681] conn(0x7f1088056040 0x7f1088056470 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:17.994 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.977+0000 7f108e2a0700 1 --2- 172.21.15.170:0/2745664170 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f1068028260 0x7f106802a720 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:17.994 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.977+0000 7f108e2a0700 1 --2- 172.21.15.170:0/2745664170 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f108811fd60 0x7f1088133bd0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:17.995 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.977+0000 7f108e2a0700 1 -- 172.21.15.170:0/2745664170 >> 172.21.15.170:0/2745664170 conn(0x7f1088091390 msgr2=0x7f1088092a80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:17.995 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.977+0000 7f108e2a0700 1 -- 172.21.15.170:0/2745664170 shutdown_connections 2020-03-19T21:18:17.995 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:17.977+0000 7f108e2a0700 1 -- 172.21.15.170:0/2745664170 wait complete. 2020-03-19T21:18:17.996 INFO:teuthology.orchestra.run.smithi170.stdout:34359738373 2020-03-19T21:18:17.996 INFO:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early tell osd.2 flush_pg_stats 2020-03-19T21:18:18.102 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.100+0000 7f8f6e7a2700 1 Processor -- start 2020-03-19T21:18:18.102 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.100+0000 7f8f6e7a2700 1 -- start start 2020-03-19T21:18:18.109 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.100+0000 7f8f6e7a2700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f8f680613f0 0x7f8f6811f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T21:18:18.109 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.100+0000 7f8f6e7a2700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f68090f10 con 0x7f8f6811fcb0 2020-03-19T21:18:18.109 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.100+0000 7f8f6e7a2700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f68092200 con 0x7f8f680613f0 2020-03-19T21:18:18.109 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.100+0000 7f8f6d7a0700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f8f680613f0 0x7f8f6811f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:18.109 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.100+0000 7f8f6d7a0700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f8f680613f0 0x7f8f6811f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:52998/0 (socket says 172.21.15.170:52998) 2020-03-19T21:18:18.109 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.100+0000 7f8f6cf9f700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f8f6811fcb0 0x7f8f681226a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:53764/0 (socket says 172.21.15.170:53764) 2020-03-19T21:18:18.110 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.100+0000 7f8f6d7a0700 1 -- 172.21.15.170:0/3776176796 learned_addr learned my addr 172.21.15.170:0/3776176796 (peer_addr_for_me v2:172.21.15.170:0/0) 2020-03-19T21:18:18.110 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.101+0000 7f8f6d7a0700 1 -- 172.21.15.170:0/3776176796 >> v1:172.21.15.170:6789/0 conn(0x7f8f6811fcb0 legacy=0x7f8f681226a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:18.110 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.101+0000 7f8f6d7a0700 1 -- 172.21.15.170:0/3776176796 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f68061bd0 con 0x7f8f680613f0 2020-03-19T21:18:18.110 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.101+0000 7f8f5bfff700 1 -- 172.21.15.170:0/3776176796 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1385438036 0 0) 0x7f8f68090f10 con 0x7f8f6811fcb0 2020-03-19T21:18:18.110 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.101+0000 7f8f6d7a0700 1 --2- 172.21.15.170:0/3776176796 >> v2:172.21.15.170:3300/0 conn(0x7f8f680613f0 0x7f8f6811f850 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f8f64002f30 tx=0x7f8f64007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:18.110 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.101+0000 7f8f6d7a0700 1 --2- 172.21.15.170:0/3776176796 >> v2:172.21.15.170:3300/0 conn(0x7f8f680613f0 0x7f8f6811f850 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f8f64002f30 tx=0x7f8f64007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:18.111 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.101+0000 7f8f6d7a0700 1 --2- 172.21.15.170:0/3776176796 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8f680613f0 0x7f8f6811f850 secure :-1 s=READY pgs=147 cs=0 l=1 rx=0x7f8f64002f30 tx=0x7f8f64007a40).ready entity=mon.0 client_cookie=effc4106f4352953 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:18.111 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.101+0000 7f8f6d7a0700 1 --2- 172.21.15.170:0/3776176796 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8f680613f0 0x7f8f6811f850 secure :-1 s=THROTTLE_DONE pgs=147 cs=0 l=1 rx=0x7f8f64002f30 tx=0x7f8f64007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:18.111 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.101+0000 7f8f5bfff700 1 -- 172.21.15.170:0/3776176796 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f8f640114b0 con 0x7f8f680613f0 2020-03-19T21:18:18.111 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.101+0000 7f8f6d7a0700 1 --2- 172.21.15.170:0/3776176796 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8f680613f0 0x7f8f6811f850 secure :-1 s=THROTTLE_DONE pgs=147 cs=0 l=1 rx=0x7f8f64002f30 tx=0x7f8f64007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:18.111 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.101+0000 7f8f6d7a0700 1 --2- 172.21.15.170:0/3776176796 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8f680613f0 0x7f8f6811f850 secure :-1 s=THROTTLE_DONE pgs=147 cs=0 l=1 rx=0x7f8f64002f30 tx=0x7f8f64007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:18.111 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.101+0000 7f8f5bfff700 1 -- 172.21.15.170:0/3776176796 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f64011610 con 0x7f8f680613f0 2020-03-19T21:18:18.112 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.101+0000 7f8f5bfff700 1 -- 172.21.15.170:0/3776176796 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f8f64011930 con 0x7f8f680613f0 2020-03-19T21:18:18.112 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.101+0000 7f8f6e7a2700 1 -- 172.21.15.170:0/3776176796 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8f680613f0 msgr2=0x7f8f6811f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:18.112 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.101+0000 7f8f6e7a2700 1 --2- 172.21.15.170:0/3776176796 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8f680613f0 0x7f8f6811f850 secure :-1 s=READY pgs=147 cs=0 l=1 rx=0x7f8f64002f30 tx=0x7f8f64007a40).stop 2020-03-19T21:18:18.112 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.101+0000 7f8f6e7a2700 1 -- 172.21.15.170:0/3776176796 shutdown_connections 2020-03-19T21:18:18.112 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.101+0000 7f8f6e7a2700 1 --2- 172.21.15.170:0/3776176796 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8f680613f0 0x7f8f6811f850 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:18.112 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.101+0000 7f8f6e7a2700 1 -- 172.21.15.170:0/3776176796 >> 172.21.15.170:0/3776176796 conn(0x7f8f68091390 msgr2=0x7f8f68091770 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:18.112 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.101+0000 7f8f6e7a2700 1 -- 172.21.15.170:0/3776176796 shutdown_connections 2020-03-19T21:18:18.113 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.101+0000 7f8f6e7a2700 1 -- 172.21.15.170:0/3776176796 wait complete. 2020-03-19T21:18:18.113 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.102+0000 7f8f6e7a2700 1 Processor -- start 2020-03-19T21:18:18.113 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.102+0000 7f8f6e7a2700 1 -- start start 2020-03-19T21:18:18.113 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.103+0000 7f8f6e7a2700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f8f680613f0 0x7f8f6811b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:18.113 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.103+0000 7f8f6e7a2700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f6810ba50 con 0x7f8f6811fcb0 2020-03-19T21:18:18.113 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.103+0000 7f8f6e7a2700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f68066500 con 0x7f8f680613f0 2020-03-19T21:18:18.113 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.103+0000 7f8f6d7a0700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f8f680613f0 0x7f8f6811b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:18.114 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.103+0000 7f8f6cf9f700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f8f6811fcb0 0x7f8f6811bcb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:53768/0 (socket says 172.21.15.170:53768) 2020-03-19T21:18:18.114 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.103+0000 7f8f6d7a0700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f8f680613f0 0x7f8f6811b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53002/0 (socket says 172.21.15.170:53002) 2020-03-19T21:18:18.114 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.103+0000 7f8f6cf9f700 1 -- 172.21.15.170:0/2781699387 learned_addr learned my addr 172.21.15.170:0/2781699387 (peer_addr_for_me v1:172.21.15.170:0/0) 2020-03-19T21:18:18.114 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.103+0000 7f8f6d7a0700 1 -- 172.21.15.170:0/2781699387 >> v1:172.21.15.170:6789/0 conn(0x7f8f6811fcb0 legacy=0x7f8f6811bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:18.114 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.103+0000 7f8f5a7fc700 1 -- 172.21.15.170:0/2781699387 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4054114075 0 0) 0x7f8f6810ba50 con 0x7f8f6811fcb0 2020-03-19T21:18:18.114 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.103+0000 7f8f6d7a0700 1 -- 172.21.15.170:0/2781699387 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f6810b7f0 con 0x7f8f680613f0 2020-03-19T21:18:18.114 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.103+0000 7f8f6d7a0700 1 --2- 172.21.15.170:0/2781699387 >> v2:172.21.15.170:3300/0 conn(0x7f8f680613f0 0x7f8f6811b850 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f8f68066f20 tx=0x7f8f64007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:18.115 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.103+0000 7f8f6d7a0700 1 --2- 172.21.15.170:0/2781699387 >> v2:172.21.15.170:3300/0 conn(0x7f8f680613f0 0x7f8f6811b850 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f8f68066f20 tx=0x7f8f64007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:18.115 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.103+0000 7f8f6d7a0700 1 --2- 172.21.15.170:0/2781699387 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8f680613f0 0x7f8f6811b850 secure :-1 s=READY pgs=149 cs=0 l=1 rx=0x7f8f68066f20 tx=0x7f8f64007a40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:18.115 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.103+0000 7f8f6d7a0700 1 --2- 172.21.15.170:0/2781699387 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8f680613f0 0x7f8f6811b850 secure :-1 s=THROTTLE_DONE pgs=149 cs=0 l=1 rx=0x7f8f68066f20 tx=0x7f8f64007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:18.115 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.103+0000 7f8f5a7fc700 1 -- 172.21.15.170:0/2781699387 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f8f6400f830 con 0x7f8f680613f0 2020-03-19T21:18:18.115 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.103+0000 7f8f6d7a0700 1 --2- 172.21.15.170:0/2781699387 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8f680613f0 0x7f8f6811b850 secure :-1 s=THROTTLE_DONE pgs=149 cs=0 l=1 rx=0x7f8f68066f20 tx=0x7f8f64007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:18.115 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.103+0000 7f8f6d7a0700 1 --2- 172.21.15.170:0/2781699387 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8f680613f0 0x7f8f6811b850 secure :-1 s=THROTTLE_DONE pgs=149 cs=0 l=1 rx=0x7f8f68066f20 tx=0x7f8f64007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:18.115 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.103+0000 7f8f5a7fc700 1 -- 172.21.15.170:0/2781699387 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f64007150 con 0x7f8f680613f0 2020-03-19T21:18:18.115 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.103+0000 7f8f6e7a2700 1 -- 172.21.15.170:0/2781699387 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f6810d570 con 0x7f8f680613f0 2020-03-19T21:18:18.116 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.103+0000 7f8f5a7fc700 1 -- 172.21.15.170:0/2781699387 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f8f640073f0 con 0x7f8f680613f0 2020-03-19T21:18:18.116 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.103+0000 7f8f6e7a2700 1 -- 172.21.15.170:0/2781699387 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f6811c3c0 con 0x7f8f680613f0 2020-03-19T21:18:18.116 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.104+0000 7f8f6d7a0700 1 --2- 172.21.15.170:0/2781699387 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8f680613f0 0x7f8f6811b850 secure :-1 s=THROTTLE_DONE pgs=149 cs=0 l=1 rx=0x7f8f68066f20 tx=0x7f8f64007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:18.116 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.104+0000 7f8f5a7fc700 1 -- 172.21.15.170:0/2781699387 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 4) v1 ==== 29603+0+0 (secure 0 0 0) 0x7f8f640056c0 con 0x7f8f680613f0 2020-03-19T21:18:18.116 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.104+0000 7f8f6d7a0700 1 --2- 172.21.15.170:0/2781699387 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8f680613f0 0x7f8f6811b850 secure :-1 s=THROTTLE_DONE pgs=149 cs=0 l=1 rx=0x7f8f68066f20 tx=0x7f8f64007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:18.116 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.104+0000 7f8f5a7fc700 1 --2- 172.21.15.170:0/2781699387 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f8f48028230 0x7f8f4802a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:18.116 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.104+0000 7f8f5a7fc700 1 -- 172.21.15.170:0/2781699387 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3403+0+0 (secure 0 0 0) 0x7f8f64034850 con 0x7f8f680613f0 2020-03-19T21:18:18.117 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.104+0000 7f8f6cf9f700 1 --2- 172.21.15.170:0/2781699387 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f8f48028230 0x7f8f4802a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:18.117 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.104+0000 7f8f6e7a2700 1 --2- 172.21.15.170:0/2781699387 >> [v2:172.21.15.170:6816/20682,v1:172.21.15.170:6817/20682] conn(0x7f8f68056040 0x7f8f68056450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:18.117 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.104+0000 7f8f6e7a2700 1 -- 172.21.15.170:0/2781699387 --> [v2:172.21.15.170:6816/20682,v1:172.21.15.170:6817/20682] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8f68066ba0 con 0x7f8f68056040 2020-03-19T21:18:18.117 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.104+0000 7f8f6cf9f700 1 --2- 172.21.15.170:0/2781699387 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f8f48028230 0x7f8f4802a6f0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f8f5c0012f0 tx=0x7f8f5c0017b0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:18.117 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.104+0000 7f8f6dfa1700 1 --2- 172.21.15.170:0/2781699387 >> [v2:172.21.15.170:6816/20682,v1:172.21.15.170:6817/20682] conn(0x7f8f68056040 0x7f8f68056450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:18.117 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.104+0000 7f8f6cf9f700 1 --2- 172.21.15.170:0/2781699387 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f8f48028230 0x7f8f4802a6f0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f8f5c0012f0 tx=0x7f8f5c0017b0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:18.117 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.104+0000 7f8f6cf9f700 1 --2- 172.21.15.170:0/2781699387 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f8f48028230 0x7f8f4802a6f0 secure :-1 s=READY pgs=37 cs=0 l=1 rx=0x7f8f5c0012f0 tx=0x7f8f5c0017b0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:18.118 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.105+0000 7f8f6dfa1700 1 --2- 172.21.15.170:0/2781699387 >> [v2:172.21.15.170:6816/20682,v1:172.21.15.170:6817/20682] conn(0x7f8f68056040 0x7f8f68056450 crc :-1 s=READY pgs=7 cs=0 l=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:18.118 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.105+0000 7f8f5a7fc700 1 -- 172.21.15.170:0/2781699387 <== osd.2 v2:172.21.15.170:6816/20682 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23288 (crc 0 0 0) 0x7f8f68066ba0 con 0x7f8f68056040 2020-03-19T21:18:18.125 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.123+0000 7f8f6e7a2700 1 -- 172.21.15.170:0/2781699387 --> [v2:172.21.15.170:6816/20682,v1:172.21.15.170:6817/20682] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f8f680568b0 con 0x7f8f68056040 2020-03-19T21:18:18.126 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.123+0000 7f8f5a7fc700 1 -- 172.21.15.170:0/2781699387 <== osd.2 v2:172.21.15.170:6816/20682 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f8f680568b0 con 0x7f8f68056040 2020-03-19T21:18:18.126 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.123+0000 7f8f6e7a2700 1 -- 172.21.15.170:0/2781699387 >> [v2:172.21.15.170:6816/20682,v1:172.21.15.170:6817/20682] conn(0x7f8f68056040 msgr2=0x7f8f68056450 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:18.126 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.123+0000 7f8f6e7a2700 1 --2- 172.21.15.170:0/2781699387 >> [v2:172.21.15.170:6816/20682,v1:172.21.15.170:6817/20682] conn(0x7f8f68056040 0x7f8f68056450 crc :-1 s=READY pgs=7 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:18.127 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.123+0000 7f8f6e7a2700 1 -- 172.21.15.170:0/2781699387 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f8f48028230 msgr2=0x7f8f4802a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:18.127 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.123+0000 7f8f6e7a2700 1 --2- 172.21.15.170:0/2781699387 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f8f48028230 0x7f8f4802a6f0 secure :-1 s=READY pgs=37 cs=0 l=1 rx=0x7f8f5c0012f0 tx=0x7f8f5c0017b0).stop 2020-03-19T21:18:18.127 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.123+0000 7f8f6e7a2700 1 -- 172.21.15.170:0/2781699387 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8f680613f0 msgr2=0x7f8f6811b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:18.127 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.123+0000 7f8f6e7a2700 1 --2- 172.21.15.170:0/2781699387 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8f680613f0 0x7f8f6811b850 secure :-1 s=READY pgs=149 cs=0 l=1 rx=0x7f8f68066f20 tx=0x7f8f64007a40).stop 2020-03-19T21:18:18.127 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.124+0000 7f8f6e7a2700 1 -- 172.21.15.170:0/2781699387 shutdown_connections 2020-03-19T21:18:18.127 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.124+0000 7f8f6e7a2700 1 --2- 172.21.15.170:0/2781699387 >> [v2:172.21.15.170:6816/20682,v1:172.21.15.170:6817/20682] conn(0x7f8f68056040 0x7f8f68056450 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:18.127 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.124+0000 7f8f6e7a2700 1 --2- 172.21.15.170:0/2781699387 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f8f48028230 0x7f8f4802a6f0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:18.128 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.124+0000 7f8f6e7a2700 1 --2- 172.21.15.170:0/2781699387 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8f680613f0 0x7f8f6811b850 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:18.128 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.124+0000 7f8f6e7a2700 1 -- 172.21.15.170:0/2781699387 >> 172.21.15.170:0/2781699387 conn(0x7f8f68091390 msgr2=0x7f8f681209f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:18.128 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.124+0000 7f8f6e7a2700 1 -- 172.21.15.170:0/2781699387 shutdown_connections 2020-03-19T21:18:18.128 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.124+0000 7f8f6e7a2700 1 -- 172.21.15.170:0/2781699387 wait complete. 2020-03-19T21:18:18.135 INFO:teuthology.orchestra.run.smithi170.stdout:34359738374 2020-03-19T21:18:18.136 INFO:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd last-stat-seq osd.0 2020-03-19T21:18:18.244 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.241+0000 7f78511e7700 1 Processor -- start 2020-03-19T21:18:18.244 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.241+0000 7f78511e7700 1 -- start start 2020-03-19T21:18:18.249 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.242+0000 7f78511e7700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f784c091eb0 0x7f784c11f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T21:18:18.249 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.242+0000 7f78511e7700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f784c0911c0 con 0x7f784c11fab0 2020-03-19T21:18:18.249 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.242+0000 7f78511e7700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f784c067130 con 0x7f784c091eb0 2020-03-19T21:18:18.250 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.242+0000 7f784ad9d700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f784c091eb0 0x7f784c11f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:18.250 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.242+0000 7f784ad9d700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f784c091eb0 0x7f784c11f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53010/0 (socket says 172.21.15.170:53010) 2020-03-19T21:18:18.250 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.242+0000 7f784ad9d700 1 -- 172.21.15.170:0/3600237270 learned_addr learned my addr 172.21.15.170:0/3600237270 (peer_addr_for_me v2:172.21.15.170:0/0) 2020-03-19T21:18:18.250 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.242+0000 7f784a59c700 1 --1- 172.21.15.170:0/3600237270 >> v1:172.21.15.170:6789/0 conn(0x7f784c11fab0 0x7f784c060110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:53774/0 (socket says 172.21.15.170:53774) 2020-03-19T21:18:18.250 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.242+0000 7f784ad9d700 1 -- 172.21.15.170:0/3600237270 >> v1:172.21.15.170:6789/0 conn(0x7f784c11fab0 legacy=0x7f784c060110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:18.250 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.242+0000 7f784ad9d700 1 -- 172.21.15.170:0/3600237270 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f784c11f340 con 0x7f784c091eb0 2020-03-19T21:18:18.250 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.242+0000 7f784ad9d700 1 --2- 172.21.15.170:0/3600237270 >> v2:172.21.15.170:3300/0 conn(0x7f784c091eb0 0x7f784c11f650 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f78340010d0 tx=0x7f783c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:18.250 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.243+0000 7f784ad9d700 1 --2- 172.21.15.170:0/3600237270 >> v2:172.21.15.170:3300/0 conn(0x7f784c091eb0 0x7f784c11f650 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f78340010d0 tx=0x7f783c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:18.251 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.243+0000 7f784ad9d700 1 --2- 172.21.15.170:0/3600237270 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f784c091eb0 0x7f784c11f650 secure :-1 s=READY pgs=151 cs=0 l=1 rx=0x7f78340010d0 tx=0x7f783c007a40).ready entity=mon.0 client_cookie=9f4a84e52d8e49c7 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:18.251 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.243+0000 7f784ad9d700 1 --2- 172.21.15.170:0/3600237270 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f784c091eb0 0x7f784c11f650 secure :-1 s=THROTTLE_DONE pgs=151 cs=0 l=1 rx=0x7f78340010d0 tx=0x7f783c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:18.251 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.243+0000 7f7849d9b700 1 -- 172.21.15.170:0/3600237270 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f783c008c90 con 0x7f784c091eb0 2020-03-19T21:18:18.251 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.243+0000 7f784ad9d700 1 --2- 172.21.15.170:0/3600237270 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f784c091eb0 0x7f784c11f650 secure :-1 s=THROTTLE_DONE pgs=151 cs=0 l=1 rx=0x7f78340010d0 tx=0x7f783c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:18.251 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.243+0000 7f7849d9b700 1 -- 172.21.15.170:0/3600237270 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f783c008df0 con 0x7f784c091eb0 2020-03-19T21:18:18.251 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.243+0000 7f784ad9d700 1 --2- 172.21.15.170:0/3600237270 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f784c091eb0 0x7f784c11f650 secure :-1 s=THROTTLE_DONE pgs=151 cs=0 l=1 rx=0x7f78340010d0 tx=0x7f783c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:18.251 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.243+0000 7f78511e7700 1 -- 172.21.15.170:0/3600237270 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f784c091eb0 msgr2=0x7f784c11f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:18.252 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.243+0000 7f7849d9b700 1 -- 172.21.15.170:0/3600237270 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f783c011950 con 0x7f784c091eb0 2020-03-19T21:18:18.252 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.243+0000 7f78511e7700 1 --2- 172.21.15.170:0/3600237270 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f784c091eb0 0x7f784c11f650 secure :-1 s=READY pgs=151 cs=0 l=1 rx=0x7f78340010d0 tx=0x7f783c007a40).stop 2020-03-19T21:18:18.252 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.243+0000 7f78511e7700 1 -- 172.21.15.170:0/3600237270 shutdown_connections 2020-03-19T21:18:18.252 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.243+0000 7f78511e7700 1 --2- 172.21.15.170:0/3600237270 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f784c091eb0 0x7f784c11f650 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:18.252 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.243+0000 7f78511e7700 1 -- 172.21.15.170:0/3600237270 >> 172.21.15.170:0/3600237270 conn(0x7f784c091390 msgr2=0x7f784c0917a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:18.252 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.243+0000 7f78511e7700 1 -- 172.21.15.170:0/3600237270 shutdown_connections 2020-03-19T21:18:18.252 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.243+0000 7f78511e7700 1 -- 172.21.15.170:0/3600237270 wait complete. 2020-03-19T21:18:18.253 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.244+0000 7f78511e7700 1 Processor -- start 2020-03-19T21:18:18.253 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.244+0000 7f78511e7700 1 -- start start 2020-03-19T21:18:18.253 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.244+0000 7f78511e7700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f784c11fab0 0x7f784c066800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:18.253 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.244+0000 7f78511e7700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f784c10d860 con 0x7f784c091eb0 2020-03-19T21:18:18.253 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.244+0000 7f78511e7700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f784c10bf70 con 0x7f784c11fab0 2020-03-19T21:18:18.253 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.244+0000 7f784ad9d700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f784c091eb0 0x7f784c130e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:53778/0 (socket says 172.21.15.170:53778) 2020-03-19T21:18:18.253 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.244+0000 7f784ad9d700 1 -- 172.21.15.170:0/4070609822 learned_addr learned my addr 172.21.15.170:0/4070609822 (peer_addr_for_me v1:172.21.15.170:0/0) 2020-03-19T21:18:18.253 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.244+0000 7f784a59c700 1 --2- 172.21.15.170:0/4070609822 >> v2:172.21.15.170:3300/0 conn(0x7f784c11fab0 0x7f784c066800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:18.254 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.245+0000 7f7843fff700 1 -- 172.21.15.170:0/4070609822 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 276459050 0 0) 0x7f784c10d860 con 0x7f784c091eb0 2020-03-19T21:18:18.254 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.245+0000 7f7843fff700 1 -- 172.21.15.170:0/4070609822 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7830002fe0 con 0x7f784c091eb0 2020-03-19T21:18:18.254 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.245+0000 7f784a59c700 1 -- 172.21.15.170:0/4070609822 >> v1:172.21.15.170:6789/0 conn(0x7f784c091eb0 legacy=0x7f784c130e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:18.254 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.245+0000 7f784a59c700 1 -- 172.21.15.170:0/4070609822 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f784c130b60 con 0x7f784c11fab0 2020-03-19T21:18:18.254 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.245+0000 7f784a59c700 1 --2- 172.21.15.170:0/4070609822 >> v2:172.21.15.170:3300/0 conn(0x7f784c11fab0 0x7f784c066800 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f7834001190 tx=0x7f783400c480).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:18.254 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.245+0000 7f784a59c700 1 --2- 172.21.15.170:0/4070609822 >> v2:172.21.15.170:3300/0 conn(0x7f784c11fab0 0x7f784c066800 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f7834001190 tx=0x7f783400c480).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:18.254 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.245+0000 7f784a59c700 1 --2- 172.21.15.170:0/4070609822 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f784c11fab0 0x7f784c066800 secure :-1 s=READY pgs=153 cs=0 l=1 rx=0x7f7834001190 tx=0x7f783400c480).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:18.255 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.245+0000 7f784a59c700 1 --2- 172.21.15.170:0/4070609822 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f784c11fab0 0x7f784c066800 secure :-1 s=THROTTLE_DONE pgs=153 cs=0 l=1 rx=0x7f7834001190 tx=0x7f783400c480).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:18.255 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.245+0000 7f7843fff700 1 -- 172.21.15.170:0/4070609822 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f783400c890 con 0x7f784c11fab0 2020-03-19T21:18:18.255 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.245+0000 7f784a59c700 1 --2- 172.21.15.170:0/4070609822 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f784c11fab0 0x7f784c066800 secure :-1 s=THROTTLE_DONE pgs=153 cs=0 l=1 rx=0x7f7834001190 tx=0x7f783400c480).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:18.255 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.245+0000 7f784a59c700 1 --2- 172.21.15.170:0/4070609822 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f784c11fab0 0x7f784c066800 secure :-1 s=THROTTLE_DONE pgs=153 cs=0 l=1 rx=0x7f7834001190 tx=0x7f783400c480).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:18.255 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.245+0000 7f7843fff700 1 -- 172.21.15.170:0/4070609822 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f783400c9f0 con 0x7f784c11fab0 2020-03-19T21:18:18.255 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.245+0000 7f7843fff700 1 -- 172.21.15.170:0/4070609822 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f783400cd10 con 0x7f784c11fab0 2020-03-19T21:18:18.255 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.245+0000 7f78511e7700 1 -- 172.21.15.170:0/4070609822 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f784c10c890 con 0x7f784c11fab0 2020-03-19T21:18:18.256 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.245+0000 7f78511e7700 1 -- 172.21.15.170:0/4070609822 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f784c131580 con 0x7f784c11fab0 2020-03-19T21:18:18.256 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.245+0000 7f784a59c700 1 --2- 172.21.15.170:0/4070609822 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f784c11fab0 0x7f784c066800 secure :-1 s=THROTTLE_DONE pgs=153 cs=0 l=1 rx=0x7f7834001190 tx=0x7f783400c480).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:18.256 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.246+0000 7f7843fff700 1 -- 172.21.15.170:0/4070609822 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 4) v1 ==== 29603+0+0 (secure 0 0 0) 0x7f78340044f0 con 0x7f784c11fab0 2020-03-19T21:18:18.256 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.246+0000 7f784a59c700 1 --2- 172.21.15.170:0/4070609822 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f784c11fab0 0x7f784c066800 secure :-1 s=THROTTLE_DONE pgs=153 cs=0 l=1 rx=0x7f7834001190 tx=0x7f783400c480).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:18.256 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.246+0000 7f7843fff700 1 --2- 172.21.15.170:0/4070609822 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f7830028790 0x7f783002ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:18.256 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.246+0000 7f7843fff700 1 -- 172.21.15.170:0/4070609822 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3403+0+0 (secure 0 0 0) 0x7f7834033d60 con 0x7f784c11fab0 2020-03-19T21:18:18.256 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.246+0000 7f784ad9d700 1 --2- 172.21.15.170:0/4070609822 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f7830028790 0x7f783002ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:18.257 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.246+0000 7f78511e7700 1 -- 172.21.15.170:0/4070609822 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f784c10bd70 con 0x7f784c11fab0 2020-03-19T21:18:18.257 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.246+0000 7f784ad9d700 1 --2- 172.21.15.170:0/4070609822 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f7830028790 0x7f783002ac50 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f784c069390 tx=0x7f783c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:18.257 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.246+0000 7f784ad9d700 1 --2- 172.21.15.170:0/4070609822 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f7830028790 0x7f783002ac50 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f784c069390 tx=0x7f783c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:18.257 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.246+0000 7f784ad9d700 1 --2- 172.21.15.170:0/4070609822 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f7830028790 0x7f783002ac50 secure :-1 s=READY pgs=38 cs=0 l=1 rx=0x7f784c069390 tx=0x7f783c007a40).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:18.258 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.249+0000 7f784a59c700 1 --2- 172.21.15.170:0/4070609822 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f784c11fab0 0x7f784c066800 secure :-1 s=THROTTLE_DONE pgs=153 cs=0 l=1 rx=0x7f7834001190 tx=0x7f783400c480).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:18.258 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.249+0000 7f7843fff700 1 -- 172.21.15.170:0/4070609822 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+126772 (secure 0 0 0) 0x7f7834004780 con 0x7f784c11fab0 2020-03-19T21:18:18.390 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.388+0000 7f78511e7700 1 -- 172.21.15.170:0/4070609822 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f784c056db0 con 0x7f784c11fab0 2020-03-19T21:18:18.391 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.388+0000 7f784a59c700 1 --2- 172.21.15.170:0/4070609822 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f784c11fab0 0x7f784c066800 secure :-1 s=THROTTLE_DONE pgs=153 cs=0 l=1 rx=0x7f7834001190 tx=0x7f783400c480).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:18.391 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.388+0000 7f7843fff700 1 -- 172.21.15.170:0/4070609822 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f78340117d0 con 0x7f784c11fab0 2020-03-19T21:18:18.391 INFO:teuthology.orchestra.run.smithi170.stdout:34359738372 2020-03-19T21:18:18.392 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.390+0000 7f78511e7700 1 -- 172.21.15.170:0/4070609822 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f7830028790 msgr2=0x7f783002ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:18.392 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.390+0000 7f78511e7700 1 --2- 172.21.15.170:0/4070609822 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f7830028790 0x7f783002ac50 secure :-1 s=READY pgs=38 cs=0 l=1 rx=0x7f784c069390 tx=0x7f783c007a40).stop 2020-03-19T21:18:18.392 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.390+0000 7f78511e7700 1 -- 172.21.15.170:0/4070609822 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f784c11fab0 msgr2=0x7f784c066800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:18.392 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.390+0000 7f78511e7700 1 --2- 172.21.15.170:0/4070609822 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f784c11fab0 0x7f784c066800 secure :-1 s=READY pgs=153 cs=0 l=1 rx=0x7f7834001190 tx=0x7f783400c480).stop 2020-03-19T21:18:18.392 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.390+0000 7f78511e7700 1 -- 172.21.15.170:0/4070609822 shutdown_connections 2020-03-19T21:18:18.393 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.390+0000 7f78511e7700 1 --2- 172.21.15.170:0/4070609822 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f7830028790 0x7f783002ac50 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:18.393 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.390+0000 7f78511e7700 1 --2- 172.21.15.170:0/4070609822 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f784c11fab0 0x7f784c066800 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:18.393 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.390+0000 7f78511e7700 1 -- 172.21.15.170:0/4070609822 >> 172.21.15.170:0/4070609822 conn(0x7f784c091390 msgr2=0x7f784c11e470 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:18.393 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.390+0000 7f78511e7700 1 -- 172.21.15.170:0/4070609822 shutdown_connections 2020-03-19T21:18:18.393 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:18.390+0000 7f78511e7700 1 -- 172.21.15.170:0/4070609822 wait complete. 2020-03-19T21:18:18.402 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738373 got 34359738372 for osd.0 2020-03-19T21:18:19.402 INFO:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd last-stat-seq osd.0 2020-03-19T21:18:19.519 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.517+0000 7f553cbb7700 1 Processor -- start 2020-03-19T21:18:19.520 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.517+0000 7f553cbb7700 1 -- start start 2020-03-19T21:18:19.524 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.518+0000 7f553cbb7700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f55380663e0 0x7f5538127280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T21:18:19.525 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.518+0000 7f553cbb7700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55380604a0 con 0x7f55380658f0 2020-03-19T21:18:19.525 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.518+0000 7f553cbb7700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5538060810 con 0x7f55380663e0 2020-03-19T21:18:19.525 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.518+0000 7f5535d9b700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f55380663e0 0x7f5538127280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:19.525 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.518+0000 7f5535d9b700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f55380663e0 0x7f5538127280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53022/0 (socket says 172.21.15.170:53022) 2020-03-19T21:18:19.525 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.518+0000 7f553659c700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f55380658f0 0x7f5538065cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:53784/0 (socket says 172.21.15.170:53784) 2020-03-19T21:18:19.525 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.518+0000 7f5535d9b700 1 -- 172.21.15.170:0/4149598035 learned_addr learned my addr 172.21.15.170:0/4149598035 (peer_addr_for_me v2:172.21.15.170:0/0) 2020-03-19T21:18:19.525 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.518+0000 7f5535d9b700 1 -- 172.21.15.170:0/4149598035 >> v1:172.21.15.170:6789/0 conn(0x7f55380658f0 legacy=0x7f5538065cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:19.526 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.518+0000 7f553559a700 1 -- 172.21.15.170:0/4149598035 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3851133611 0 0) 0x7f55380604a0 con 0x7f55380658f0 2020-03-19T21:18:19.526 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.518+0000 7f5535d9b700 1 -- 172.21.15.170:0/4149598035 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5538060110 con 0x7f55380663e0 2020-03-19T21:18:19.526 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.518+0000 7f5535d9b700 1 --2- 172.21.15.170:0/4149598035 >> v2:172.21.15.170:3300/0 conn(0x7f55380663e0 0x7f5538127280 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f5528002f30 tx=0x7f5528007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:19.526 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.518+0000 7f5535d9b700 1 --2- 172.21.15.170:0/4149598035 >> v2:172.21.15.170:3300/0 conn(0x7f55380663e0 0x7f5538127280 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f5528002f30 tx=0x7f5528007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:19.526 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.518+0000 7f5535d9b700 1 --2- 172.21.15.170:0/4149598035 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f55380663e0 0x7f5538127280 secure :-1 s=READY pgs=155 cs=0 l=1 rx=0x7f5528002f30 tx=0x7f5528007a40).ready entity=mon.0 client_cookie=374ff65a682e41b2 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:19.526 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.518+0000 7f5535d9b700 1 --2- 172.21.15.170:0/4149598035 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f55380663e0 0x7f5538127280 secure :-1 s=THROTTLE_DONE pgs=155 cs=0 l=1 rx=0x7f5528002f30 tx=0x7f5528007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:19.526 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.518+0000 7f553559a700 1 -- 172.21.15.170:0/4149598035 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f55280114b0 con 0x7f55380663e0 2020-03-19T21:18:19.527 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.519+0000 7f5535d9b700 1 --2- 172.21.15.170:0/4149598035 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f55380663e0 0x7f5538127280 secure :-1 s=THROTTLE_DONE pgs=155 cs=0 l=1 rx=0x7f5528002f30 tx=0x7f5528007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:19.527 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.519+0000 7f5535d9b700 1 --2- 172.21.15.170:0/4149598035 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f55380663e0 0x7f5538127280 secure :-1 s=THROTTLE_DONE pgs=155 cs=0 l=1 rx=0x7f5528002f30 tx=0x7f5528007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:19.527 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.519+0000 7f553559a700 1 -- 172.21.15.170:0/4149598035 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5528011610 con 0x7f55380663e0 2020-03-19T21:18:19.527 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.519+0000 7f553559a700 1 -- 172.21.15.170:0/4149598035 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f5528011930 con 0x7f55380663e0 2020-03-19T21:18:19.527 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.519+0000 7f553cbb7700 1 -- 172.21.15.170:0/4149598035 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f55380663e0 msgr2=0x7f5538127280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:19.527 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.519+0000 7f553cbb7700 1 --2- 172.21.15.170:0/4149598035 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f55380663e0 0x7f5538127280 secure :-1 s=READY pgs=155 cs=0 l=1 rx=0x7f5528002f30 tx=0x7f5528007a40).stop 2020-03-19T21:18:19.527 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.519+0000 7f553cbb7700 1 -- 172.21.15.170:0/4149598035 shutdown_connections 2020-03-19T21:18:19.528 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.519+0000 7f553cbb7700 1 --2- 172.21.15.170:0/4149598035 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f55380663e0 0x7f5538127280 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:19.528 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.519+0000 7f553cbb7700 1 -- 172.21.15.170:0/4149598035 >> 172.21.15.170:0/4149598035 conn(0x7f5538091390 msgr2=0x7f55380917a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:19.528 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.519+0000 7f553cbb7700 1 -- 172.21.15.170:0/4149598035 shutdown_connections 2020-03-19T21:18:19.528 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.519+0000 7f553cbb7700 1 -- 172.21.15.170:0/4149598035 wait complete. 2020-03-19T21:18:19.528 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.520+0000 7f553cbb7700 1 Processor -- start 2020-03-19T21:18:19.528 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.520+0000 7f553cbb7700 1 -- start start 2020-03-19T21:18:19.528 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.520+0000 7f553cbb7700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f55380658f0 0x7f5538127be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:19.529 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.520+0000 7f553cbb7700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5538113840 con 0x7f55380663e0 2020-03-19T21:18:19.529 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.520+0000 7f553cbb7700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55381286f0 con 0x7f55380658f0 2020-03-19T21:18:19.529 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.520+0000 7f5535d9b700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f55380663e0 0x7f5538130e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:53788/0 (socket says 172.21.15.170:53788) 2020-03-19T21:18:19.529 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.520+0000 7f553659c700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f55380658f0 0x7f5538127be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:19.529 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.520+0000 7f5535d9b700 1 -- 172.21.15.170:0/1341326529 learned_addr learned my addr 172.21.15.170:0/1341326529 (peer_addr_for_me v1:172.21.15.170:0/0) 2020-03-19T21:18:19.529 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.521+0000 7f553659c700 1 -- 172.21.15.170:0/1341326529 >> v1:172.21.15.170:6789/0 conn(0x7f55380663e0 legacy=0x7f5538130e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:19.530 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.521+0000 7f553659c700 1 -- 172.21.15.170:0/1341326529 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5538128bf0 con 0x7f55380658f0 2020-03-19T21:18:19.530 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.521+0000 7f55277fe700 1 -- 172.21.15.170:0/1341326529 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3280676492 0 0) 0x7f5538113840 con 0x7f55380663e0 2020-03-19T21:18:19.530 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.521+0000 7f553659c700 1 --2- 172.21.15.170:0/1341326529 >> v2:172.21.15.170:3300/0 conn(0x7f55380658f0 0x7f5538127be0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f5538113020 tx=0x7f552c0108f0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:19.530 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.521+0000 7f553659c700 1 --2- 172.21.15.170:0/1341326529 >> v2:172.21.15.170:3300/0 conn(0x7f55380658f0 0x7f5538127be0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f5538113020 tx=0x7f552c0108f0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:19.530 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.521+0000 7f553659c700 1 --2- 172.21.15.170:0/1341326529 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f55380658f0 0x7f5538127be0 secure :-1 s=READY pgs=157 cs=0 l=1 rx=0x7f5538113020 tx=0x7f552c0108f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:19.530 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.521+0000 7f553659c700 1 --2- 172.21.15.170:0/1341326529 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f55380658f0 0x7f5538127be0 secure :-1 s=THROTTLE_DONE pgs=157 cs=0 l=1 rx=0x7f5538113020 tx=0x7f552c0108f0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:19.530 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.521+0000 7f55277fe700 1 -- 172.21.15.170:0/1341326529 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f552c011440 con 0x7f55380658f0 2020-03-19T21:18:19.530 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.521+0000 7f553659c700 1 --2- 172.21.15.170:0/1341326529 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f55380658f0 0x7f5538127be0 secure :-1 s=THROTTLE_DONE pgs=157 cs=0 l=1 rx=0x7f5538113020 tx=0x7f552c0108f0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:19.531 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.521+0000 7f55277fe700 1 -- 172.21.15.170:0/1341326529 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f552c0115a0 con 0x7f55380658f0 2020-03-19T21:18:19.531 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.521+0000 7f553659c700 1 --2- 172.21.15.170:0/1341326529 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f55380658f0 0x7f5538127be0 secure :-1 s=THROTTLE_DONE pgs=157 cs=0 l=1 rx=0x7f5538113020 tx=0x7f552c0108f0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:19.531 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.521+0000 7f553cbb7700 1 -- 172.21.15.170:0/1341326529 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5538131580 con 0x7f55380658f0 2020-03-19T21:18:19.531 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.521+0000 7f55277fe700 1 -- 172.21.15.170:0/1341326529 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f552c0118c0 con 0x7f55380658f0 2020-03-19T21:18:19.531 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.521+0000 7f553cbb7700 1 -- 172.21.15.170:0/1341326529 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5538111a50 con 0x7f55380658f0 2020-03-19T21:18:19.531 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.521+0000 7f553659c700 1 --2- 172.21.15.170:0/1341326529 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f55380658f0 0x7f5538127be0 secure :-1 s=THROTTLE_DONE pgs=157 cs=0 l=1 rx=0x7f5538113020 tx=0x7f552c0108f0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:19.532 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.522+0000 7f55277fe700 1 -- 172.21.15.170:0/1341326529 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 4) v1 ==== 29603+0+0 (secure 0 0 0) 0x7f552c00b0a0 con 0x7f55380658f0 2020-03-19T21:18:19.532 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.522+0000 7f553659c700 1 --2- 172.21.15.170:0/1341326529 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f55380658f0 0x7f5538127be0 secure :-1 s=THROTTLE_DONE pgs=157 cs=0 l=1 rx=0x7f5538113020 tx=0x7f552c0108f0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:19.532 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.522+0000 7f55277fe700 1 --2- 172.21.15.170:0/1341326529 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f5514028230 0x7f551402a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:19.532 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.522+0000 7f55277fe700 1 -- 172.21.15.170:0/1341326529 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3403+0+0 (secure 0 0 0) 0x7f552c034090 con 0x7f55380658f0 2020-03-19T21:18:19.533 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.522+0000 7f5535d9b700 1 --2- 172.21.15.170:0/1341326529 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f5514028230 0x7f551402a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:19.533 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.522+0000 7f5535d9b700 1 --2- 172.21.15.170:0/1341326529 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f5514028230 0x7f551402a6f0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f5528010280 tx=0x7f5528007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:19.533 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.522+0000 7f5535d9b700 1 --2- 172.21.15.170:0/1341326529 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f5514028230 0x7f551402a6f0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f5528010280 tx=0x7f5528007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:19.533 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.522+0000 7f5535d9b700 1 --2- 172.21.15.170:0/1341326529 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f5514028230 0x7f551402a6f0 secure :-1 s=READY pgs=39 cs=0 l=1 rx=0x7f5528010280 tx=0x7f5528007a40).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:19.533 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.522+0000 7f553cbb7700 1 -- 172.21.15.170:0/1341326529 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5538056db0 con 0x7f55380658f0 2020-03-19T21:18:19.534 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.526+0000 7f553659c700 1 --2- 172.21.15.170:0/1341326529 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f55380658f0 0x7f5538127be0 secure :-1 s=THROTTLE_DONE pgs=157 cs=0 l=1 rx=0x7f5538113020 tx=0x7f552c0108f0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:19.534 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.526+0000 7f55277fe700 1 -- 172.21.15.170:0/1341326529 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+126772 (secure 0 0 0) 0x7f552c011af0 con 0x7f55380658f0 2020-03-19T21:18:19.667 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.665+0000 7f553cbb7700 1 -- 172.21.15.170:0/1341326529 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f55380693f0 con 0x7f55380658f0 2020-03-19T21:18:19.667 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.665+0000 7f553659c700 1 --2- 172.21.15.170:0/1341326529 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f55380658f0 0x7f5538127be0 secure :-1 s=THROTTLE_DONE pgs=157 cs=0 l=1 rx=0x7f5538113020 tx=0x7f552c0108f0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:19.667 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.665+0000 7f55277fe700 1 -- 172.21.15.170:0/1341326529 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f552c007160 con 0x7f55380658f0 2020-03-19T21:18:19.668 INFO:teuthology.orchestra.run.smithi170.stdout:34359738373 2020-03-19T21:18:19.669 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.666+0000 7f553cbb7700 1 -- 172.21.15.170:0/1341326529 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f5514028230 msgr2=0x7f551402a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:19.669 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.666+0000 7f553cbb7700 1 --2- 172.21.15.170:0/1341326529 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f5514028230 0x7f551402a6f0 secure :-1 s=READY pgs=39 cs=0 l=1 rx=0x7f5528010280 tx=0x7f5528007a40).stop 2020-03-19T21:18:19.669 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.666+0000 7f553cbb7700 1 -- 172.21.15.170:0/1341326529 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f55380658f0 msgr2=0x7f5538127be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:19.669 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.666+0000 7f553cbb7700 1 --2- 172.21.15.170:0/1341326529 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f55380658f0 0x7f5538127be0 secure :-1 s=READY pgs=157 cs=0 l=1 rx=0x7f5538113020 tx=0x7f552c0108f0).stop 2020-03-19T21:18:19.669 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.666+0000 7f553cbb7700 1 -- 172.21.15.170:0/1341326529 shutdown_connections 2020-03-19T21:18:19.669 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.666+0000 7f553cbb7700 1 --2- 172.21.15.170:0/1341326529 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f5514028230 0x7f551402a6f0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:19.669 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.666+0000 7f553cbb7700 1 --2- 172.21.15.170:0/1341326529 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f55380658f0 0x7f5538127be0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:19.670 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.666+0000 7f553cbb7700 1 -- 172.21.15.170:0/1341326529 >> 172.21.15.170:0/1341326529 conn(0x7f5538091390 msgr2=0x7f553808dfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:19.670 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.666+0000 7f553cbb7700 1 -- 172.21.15.170:0/1341326529 shutdown_connections 2020-03-19T21:18:19.670 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.666+0000 7f553cbb7700 1 -- 172.21.15.170:0/1341326529 wait complete. 2020-03-19T21:18:19.678 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738373 got 34359738373 for osd.0 2020-03-19T21:18:19.678 INFO:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd last-stat-seq osd.1 2020-03-19T21:18:19.789 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.784+0000 7f8bbe57f700 1 Processor -- start 2020-03-19T21:18:19.790 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.784+0000 7f8bbe57f700 1 -- start start 2020-03-19T21:18:19.790 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.785+0000 7f8bbe57f700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f8bb811fd60 0x7f8bb8127a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T21:18:19.790 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.785+0000 7f8bbe57f700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bb80911c0 con 0x7f8bb8091eb0 2020-03-19T21:18:19.790 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.785+0000 7f8bbe57f700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bb8067130 con 0x7f8bb811fd60 2020-03-19T21:18:19.791 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.785+0000 7f8bb77fe700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f8bb811fd60 0x7f8bb8127a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:19.791 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.785+0000 7f8bb77fe700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f8bb811fd60 0x7f8bb8127a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53030/0 (socket says 172.21.15.170:53030) 2020-03-19T21:18:19.791 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.785+0000 7f8bb7fff700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f8bb8091eb0 0x7f8bb811f650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:53794/0 (socket says 172.21.15.170:53794) 2020-03-19T21:18:19.791 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.785+0000 7f8bb77fe700 1 -- 172.21.15.170:0/1372635683 learned_addr learned my addr 172.21.15.170:0/1372635683 (peer_addr_for_me v2:172.21.15.170:0/0) 2020-03-19T21:18:19.791 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.785+0000 7f8bb77fe700 1 -- 172.21.15.170:0/1372635683 >> v1:172.21.15.170:6789/0 conn(0x7f8bb8091eb0 legacy=0x7f8bb811f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:19.792 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.785+0000 7f8bb6ffd700 1 -- 172.21.15.170:0/1372635683 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2536012031 0 0) 0x7f8bb80911c0 con 0x7f8bb8091eb0 2020-03-19T21:18:19.792 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.785+0000 7f8bb77fe700 1 -- 172.21.15.170:0/1372635683 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bb811f340 con 0x7f8bb811fd60 2020-03-19T21:18:19.792 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.785+0000 7f8bb77fe700 1 --2- 172.21.15.170:0/1372635683 >> v2:172.21.15.170:3300/0 conn(0x7f8bb811fd60 0x7f8bb8127a70 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f8bac002f30 tx=0x7f8bac007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:19.792 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.785+0000 7f8bb77fe700 1 --2- 172.21.15.170:0/1372635683 >> v2:172.21.15.170:3300/0 conn(0x7f8bb811fd60 0x7f8bb8127a70 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f8bac002f30 tx=0x7f8bac007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:19.793 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.785+0000 7f8bb77fe700 1 --2- 172.21.15.170:0/1372635683 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8bb811fd60 0x7f8bb8127a70 secure :-1 s=READY pgs=159 cs=0 l=1 rx=0x7f8bac002f30 tx=0x7f8bac007a40).ready entity=mon.0 client_cookie=87a7a481ffa66a45 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:19.793 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.786+0000 7f8bb77fe700 1 --2- 172.21.15.170:0/1372635683 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8bb811fd60 0x7f8bb8127a70 secure :-1 s=THROTTLE_DONE pgs=159 cs=0 l=1 rx=0x7f8bac002f30 tx=0x7f8bac007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:19.793 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.786+0000 7f8bb6ffd700 1 -- 172.21.15.170:0/1372635683 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f8bac0114b0 con 0x7f8bb811fd60 2020-03-19T21:18:19.793 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.786+0000 7f8bb77fe700 1 --2- 172.21.15.170:0/1372635683 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8bb811fd60 0x7f8bb8127a70 secure :-1 s=THROTTLE_DONE pgs=159 cs=0 l=1 rx=0x7f8bac002f30 tx=0x7f8bac007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:19.793 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.786+0000 7f8bb77fe700 1 --2- 172.21.15.170:0/1372635683 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8bb811fd60 0x7f8bb8127a70 secure :-1 s=THROTTLE_DONE pgs=159 cs=0 l=1 rx=0x7f8bac002f30 tx=0x7f8bac007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:19.794 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.786+0000 7f8bb6ffd700 1 -- 172.21.15.170:0/1372635683 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bac011610 con 0x7f8bb811fd60 2020-03-19T21:18:19.794 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.786+0000 7f8bb6ffd700 1 -- 172.21.15.170:0/1372635683 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f8bac0118f0 con 0x7f8bb811fd60 2020-03-19T21:18:19.794 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.786+0000 7f8bbe57f700 1 -- 172.21.15.170:0/1372635683 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8bb811fd60 msgr2=0x7f8bb8127a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:19.794 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.786+0000 7f8bbe57f700 1 --2- 172.21.15.170:0/1372635683 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8bb811fd60 0x7f8bb8127a70 secure :-1 s=READY pgs=159 cs=0 l=1 rx=0x7f8bac002f30 tx=0x7f8bac007a40).stop 2020-03-19T21:18:19.794 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.786+0000 7f8bbe57f700 1 -- 172.21.15.170:0/1372635683 shutdown_connections 2020-03-19T21:18:19.794 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.786+0000 7f8bbe57f700 1 --2- 172.21.15.170:0/1372635683 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8bb811fd60 0x7f8bb8127a70 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:19.795 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.786+0000 7f8bbe57f700 1 -- 172.21.15.170:0/1372635683 >> 172.21.15.170:0/1372635683 conn(0x7f8bb8091390 msgr2=0x7f8bb80917a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:19.795 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.786+0000 7f8bbe57f700 1 -- 172.21.15.170:0/1372635683 shutdown_connections 2020-03-19T21:18:19.795 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.786+0000 7f8bbe57f700 1 -- 172.21.15.170:0/1372635683 wait complete. 2020-03-19T21:18:19.795 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.787+0000 7f8bbe57f700 1 Processor -- start 2020-03-19T21:18:19.795 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.787+0000 7f8bbe57f700 1 -- start start 2020-03-19T21:18:19.800 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.788+0000 7f8bbe57f700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f8bb8091eb0 0x7f8bb8124680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:19.800 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.788+0000 7f8bbe57f700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bb8126f90 con 0x7f8bb811fd60 2020-03-19T21:18:19.800 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.788+0000 7f8bbe57f700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bb8066950 con 0x7f8bb8091eb0 2020-03-19T21:18:19.801 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.788+0000 7f8bb7fff700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f8bb8091eb0 0x7f8bb8124680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:19.801 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.788+0000 7f8bb77fe700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f8bb811fd60 0x7f8bb8125100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:53800/0 (socket says 172.21.15.170:53800) 2020-03-19T21:18:19.801 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.788+0000 7f8bb7fff700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f8bb8091eb0 0x7f8bb8124680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53034/0 (socket says 172.21.15.170:53034) 2020-03-19T21:18:19.801 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.788+0000 7f8bb77fe700 1 -- 172.21.15.170:0/1619222370 learned_addr learned my addr 172.21.15.170:0/1619222370 (peer_addr_for_me v1:172.21.15.170:0/0) 2020-03-19T21:18:19.801 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.788+0000 7f8bb7fff700 1 -- 172.21.15.170:0/1619222370 >> v1:172.21.15.170:6789/0 conn(0x7f8bb811fd60 legacy=0x7f8bb8125100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:19.801 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.788+0000 7f8bb57fa700 1 -- 172.21.15.170:0/1619222370 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1917618609 0 0) 0x7f8bb8126f90 con 0x7f8bb811fd60 2020-03-19T21:18:19.802 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.788+0000 7f8bb7fff700 1 -- 172.21.15.170:0/1619222370 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bb81202e0 con 0x7f8bb8091eb0 2020-03-19T21:18:19.802 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.788+0000 7f8bb7fff700 1 --2- 172.21.15.170:0/1619222370 >> v2:172.21.15.170:3300/0 conn(0x7f8bb8091eb0 0x7f8bb8124680 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f8ba8000b80 tx=0x7f8ba80108d0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:19.802 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.788+0000 7f8bb7fff700 1 --2- 172.21.15.170:0/1619222370 >> v2:172.21.15.170:3300/0 conn(0x7f8bb8091eb0 0x7f8bb8124680 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f8ba8000b80 tx=0x7f8ba80108d0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:19.802 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.788+0000 7f8bb7fff700 1 --2- 172.21.15.170:0/1619222370 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8bb8091eb0 0x7f8bb8124680 secure :-1 s=READY pgs=161 cs=0 l=1 rx=0x7f8ba8000b80 tx=0x7f8ba80108d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:19.802 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.788+0000 7f8bb7fff700 1 --2- 172.21.15.170:0/1619222370 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8bb8091eb0 0x7f8bb8124680 secure :-1 s=THROTTLE_DONE pgs=161 cs=0 l=1 rx=0x7f8ba8000b80 tx=0x7f8ba80108d0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:19.802 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.788+0000 7f8bb57fa700 1 -- 172.21.15.170:0/1619222370 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f8ba8011460 con 0x7f8bb8091eb0 2020-03-19T21:18:19.803 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.789+0000 7f8bb7fff700 1 --2- 172.21.15.170:0/1619222370 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8bb8091eb0 0x7f8bb8124680 secure :-1 s=THROTTLE_DONE pgs=161 cs=0 l=1 rx=0x7f8ba8000b80 tx=0x7f8ba80108d0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:19.803 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.789+0000 7f8bb7fff700 1 --2- 172.21.15.170:0/1619222370 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8bb8091eb0 0x7f8bb8124680 secure :-1 s=THROTTLE_DONE pgs=161 cs=0 l=1 rx=0x7f8ba8000b80 tx=0x7f8ba80108d0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:19.803 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.789+0000 7f8bb57fa700 1 -- 172.21.15.170:0/1619222370 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ba80115c0 con 0x7f8bb8091eb0 2020-03-19T21:18:19.803 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.789+0000 7f8bbe57f700 1 -- 172.21.15.170:0/1619222370 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bb8062bc0 con 0x7f8bb8091eb0 2020-03-19T21:18:19.803 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.789+0000 7f8bb57fa700 1 -- 172.21.15.170:0/1619222370 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f8ba80118e0 con 0x7f8bb8091eb0 2020-03-19T21:18:19.803 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.789+0000 7f8bbe57f700 1 -- 172.21.15.170:0/1619222370 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bb8062730 con 0x7f8bb8091eb0 2020-03-19T21:18:19.804 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.789+0000 7f8bb7fff700 1 --2- 172.21.15.170:0/1619222370 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8bb8091eb0 0x7f8bb8124680 secure :-1 s=THROTTLE_DONE pgs=161 cs=0 l=1 rx=0x7f8ba8000b80 tx=0x7f8ba80108d0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:19.804 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.789+0000 7f8bb57fa700 1 -- 172.21.15.170:0/1619222370 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 4) v1 ==== 29603+0+0 (secure 0 0 0) 0x7f8ba800b0a0 con 0x7f8bb8091eb0 2020-03-19T21:18:19.804 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.789+0000 7f8bb7fff700 1 --2- 172.21.15.170:0/1619222370 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8bb8091eb0 0x7f8bb8124680 secure :-1 s=THROTTLE_DONE pgs=161 cs=0 l=1 rx=0x7f8ba8000b80 tx=0x7f8ba80108d0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:19.805 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.789+0000 7f8bb57fa700 1 --2- 172.21.15.170:0/1619222370 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f8b98028280 0x7f8b9802a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:19.805 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.789+0000 7f8bb57fa700 1 -- 172.21.15.170:0/1619222370 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3403+0+0 (secure 0 0 0) 0x7f8ba8033ff0 con 0x7f8bb8091eb0 2020-03-19T21:18:19.805 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.789+0000 7f8bb77fe700 1 --2- 172.21.15.170:0/1619222370 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f8b98028280 0x7f8b9802a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:19.805 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.790+0000 7f8bbe57f700 1 -- 172.21.15.170:0/1619222370 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bb8127120 con 0x7f8bb8091eb0 2020-03-19T21:18:19.805 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.790+0000 7f8bb77fe700 1 --2- 172.21.15.170:0/1619222370 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f8b98028280 0x7f8b9802a740 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f8bac002ed0 tx=0x7f8bac007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:19.806 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.790+0000 7f8bb77fe700 1 --2- 172.21.15.170:0/1619222370 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f8b98028280 0x7f8b9802a740 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f8bac002ed0 tx=0x7f8bac007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:19.806 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.790+0000 7f8bb77fe700 1 --2- 172.21.15.170:0/1619222370 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f8b98028280 0x7f8b9802a740 secure :-1 s=READY pgs=40 cs=0 l=1 rx=0x7f8bac002ed0 tx=0x7f8bac007a40).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:19.806 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.793+0000 7f8bb7fff700 1 --2- 172.21.15.170:0/1619222370 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8bb8091eb0 0x7f8bb8124680 secure :-1 s=THROTTLE_DONE pgs=161 cs=0 l=1 rx=0x7f8ba8000b80 tx=0x7f8ba80108d0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:19.806 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.793+0000 7f8bb57fa700 1 -- 172.21.15.170:0/1619222370 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+126772 (secure 0 0 0) 0x7f8ba8011b50 con 0x7f8bb8091eb0 2020-03-19T21:18:19.933 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.931+0000 7f8bbe57f700 1 -- 172.21.15.170:0/1619222370 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f8bb8056db0 con 0x7f8bb8091eb0 2020-03-19T21:18:19.933 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.931+0000 7f8bb7fff700 1 --2- 172.21.15.170:0/1619222370 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8bb8091eb0 0x7f8bb8124680 secure :-1 s=THROTTLE_DONE pgs=161 cs=0 l=1 rx=0x7f8ba8000b80 tx=0x7f8ba80108d0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:19.934 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.931+0000 7f8bb57fa700 1 -- 172.21.15.170:0/1619222370 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8ba8007160 con 0x7f8bb8091eb0 2020-03-19T21:18:19.934 INFO:teuthology.orchestra.run.smithi170.stdout:34359738373 2020-03-19T21:18:19.935 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.932+0000 7f8bbe57f700 1 -- 172.21.15.170:0/1619222370 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f8b98028280 msgr2=0x7f8b9802a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:19.935 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.932+0000 7f8bbe57f700 1 --2- 172.21.15.170:0/1619222370 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f8b98028280 0x7f8b9802a740 secure :-1 s=READY pgs=40 cs=0 l=1 rx=0x7f8bac002ed0 tx=0x7f8bac007a40).stop 2020-03-19T21:18:19.935 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.932+0000 7f8bbe57f700 1 -- 172.21.15.170:0/1619222370 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8bb8091eb0 msgr2=0x7f8bb8124680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:19.936 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.932+0000 7f8bbe57f700 1 --2- 172.21.15.170:0/1619222370 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8bb8091eb0 0x7f8bb8124680 secure :-1 s=READY pgs=161 cs=0 l=1 rx=0x7f8ba8000b80 tx=0x7f8ba80108d0).stop 2020-03-19T21:18:19.936 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.932+0000 7f8bbe57f700 1 -- 172.21.15.170:0/1619222370 shutdown_connections 2020-03-19T21:18:19.936 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.932+0000 7f8bbe57f700 1 --2- 172.21.15.170:0/1619222370 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f8b98028280 0x7f8b9802a740 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:19.936 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.932+0000 7f8bbe57f700 1 --2- 172.21.15.170:0/1619222370 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8bb8091eb0 0x7f8bb8124680 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:19.936 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.932+0000 7f8bbe57f700 1 -- 172.21.15.170:0/1619222370 >> 172.21.15.170:0/1619222370 conn(0x7f8bb8091390 msgr2=0x7f8bb8060110 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:19.936 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.932+0000 7f8bbe57f700 1 -- 172.21.15.170:0/1619222370 shutdown_connections 2020-03-19T21:18:19.937 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:19.932+0000 7f8bbe57f700 1 -- 172.21.15.170:0/1619222370 wait complete. 2020-03-19T21:18:19.945 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738373 got 34359738373 for osd.1 2020-03-19T21:18:19.945 INFO:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd last-stat-seq osd.2 2020-03-19T21:18:20.058 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.049+0000 7fc37828a700 1 Processor -- start 2020-03-19T21:18:20.058 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.049+0000 7fc37828a700 1 -- start start 2020-03-19T21:18:20.058 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.049+0000 7fc37828a700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fc370091eb0 0x7fc37011f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T21:18:20.058 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.050+0000 7fc37828a700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3700911c0 con 0x7fc37011fab0 2020-03-19T21:18:20.058 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.050+0000 7fc37828a700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc370067130 con 0x7fc370091eb0 2020-03-19T21:18:20.059 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.050+0000 7fc376026700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fc370091eb0 0x7fc37011f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:20.059 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.050+0000 7fc375825700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fc37011fab0 0x7fc370060110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:53804/0 (socket says 172.21.15.170:53804) 2020-03-19T21:18:20.059 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.050+0000 7fc376026700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fc370091eb0 0x7fc37011f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53040/0 (socket says 172.21.15.170:53040) 2020-03-19T21:18:20.059 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.050+0000 7fc375825700 1 -- 172.21.15.170:0/2889544272 learned_addr learned my addr 172.21.15.170:0/2889544272 (peer_addr_for_me v1:172.21.15.170:0/0) 2020-03-19T21:18:20.059 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.050+0000 7fc376026700 1 -- 172.21.15.170:0/2889544272 >> v1:172.21.15.170:6789/0 conn(0x7fc37011fab0 legacy=0x7fc370060110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:20.059 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.050+0000 7fc376026700 1 -- 172.21.15.170:0/2889544272 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc37011f340 con 0x7fc370091eb0 2020-03-19T21:18:20.059 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.050+0000 7fc375024700 1 -- 172.21.15.170:0/2889544272 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1388554445 0 0) 0x7fc3700911c0 con 0x7fc37011fab0 2020-03-19T21:18:20.060 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.050+0000 7fc376026700 1 --2- 172.21.15.170:0/2889544272 >> v2:172.21.15.170:3300/0 conn(0x7fc370091eb0 0x7fc37011f650 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7fc360002f30 tx=0x7fc360007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:20.060 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.050+0000 7fc376026700 1 --2- 172.21.15.170:0/2889544272 >> v2:172.21.15.170:3300/0 conn(0x7fc370091eb0 0x7fc37011f650 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7fc360002f30 tx=0x7fc360007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:20.060 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.050+0000 7fc376026700 1 --2- 172.21.15.170:0/2889544272 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc370091eb0 0x7fc37011f650 secure :-1 s=READY pgs=163 cs=0 l=1 rx=0x7fc360002f30 tx=0x7fc360007a40).ready entity=mon.0 client_cookie=9c19fc9febcb51ea server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:20.060 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.050+0000 7fc376026700 1 --2- 172.21.15.170:0/2889544272 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc370091eb0 0x7fc37011f650 secure :-1 s=THROTTLE_DONE pgs=163 cs=0 l=1 rx=0x7fc360002f30 tx=0x7fc360007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:20.060 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.050+0000 7fc375024700 1 -- 172.21.15.170:0/2889544272 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fc360008e80 con 0x7fc370091eb0 2020-03-19T21:18:20.060 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.050+0000 7fc376026700 1 --2- 172.21.15.170:0/2889544272 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc370091eb0 0x7fc37011f650 secure :-1 s=THROTTLE_DONE pgs=163 cs=0 l=1 rx=0x7fc360002f30 tx=0x7fc360007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:20.060 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.050+0000 7fc375024700 1 -- 172.21.15.170:0/2889544272 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3600113f0 con 0x7fc370091eb0 2020-03-19T21:18:20.061 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.050+0000 7fc376026700 1 --2- 172.21.15.170:0/2889544272 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc370091eb0 0x7fc37011f650 secure :-1 s=THROTTLE_DONE pgs=163 cs=0 l=1 rx=0x7fc360002f30 tx=0x7fc360007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:20.061 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.050+0000 7fc375024700 1 -- 172.21.15.170:0/2889544272 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fc360011690 con 0x7fc370091eb0 2020-03-19T21:18:20.061 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.050+0000 7fc37828a700 1 -- 172.21.15.170:0/2889544272 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc370091eb0 msgr2=0x7fc37011f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:20.061 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.050+0000 7fc37828a700 1 --2- 172.21.15.170:0/2889544272 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc370091eb0 0x7fc37011f650 secure :-1 s=READY pgs=163 cs=0 l=1 rx=0x7fc360002f30 tx=0x7fc360007a40).stop 2020-03-19T21:18:20.061 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.051+0000 7fc37828a700 1 -- 172.21.15.170:0/2889544272 shutdown_connections 2020-03-19T21:18:20.061 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.051+0000 7fc37828a700 1 --2- 172.21.15.170:0/2889544272 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc370091eb0 0x7fc37011f650 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:20.061 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.051+0000 7fc37828a700 1 -- 172.21.15.170:0/2889544272 >> 172.21.15.170:0/2889544272 conn(0x7fc370091390 msgr2=0x7fc3700917a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:20.061 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.051+0000 7fc37828a700 1 -- 172.21.15.170:0/2889544272 shutdown_connections 2020-03-19T21:18:20.062 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.051+0000 7fc37828a700 1 -- 172.21.15.170:0/2889544272 wait complete. 2020-03-19T21:18:20.062 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.051+0000 7fc37828a700 1 Processor -- start 2020-03-19T21:18:20.062 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.051+0000 7fc37828a700 1 -- start start 2020-03-19T21:18:20.062 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.052+0000 7fc37828a700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fc370091eb0 0x7fc370064ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:20.062 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.052+0000 7fc37828a700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc37010c4f0 con 0x7fc37011fab0 2020-03-19T21:18:20.062 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.052+0000 7fc37828a700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc37010bf60 con 0x7fc370091eb0 2020-03-19T21:18:20.062 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.052+0000 7fc376026700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fc370091eb0 0x7fc370064ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:20.063 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.052+0000 7fc376026700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fc370091eb0 0x7fc370064ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53044/0 (socket says 172.21.15.170:53044) 2020-03-19T21:18:20.063 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.052+0000 7fc375825700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fc37011fab0 0x7fc370064f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:53810/0 (socket says 172.21.15.170:53810) 2020-03-19T21:18:20.063 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.052+0000 7fc376026700 1 -- 172.21.15.170:0/2819985477 learned_addr learned my addr 172.21.15.170:0/2819985477 (peer_addr_for_me v2:172.21.15.170:0/0) 2020-03-19T21:18:20.063 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.052+0000 7fc376026700 1 -- 172.21.15.170:0/2819985477 >> v1:172.21.15.170:6789/0 conn(0x7fc37011fab0 legacy=0x7fc370064f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:20.063 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.052+0000 7fc376026700 1 -- 172.21.15.170:0/2819985477 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc370065ce0 con 0x7fc370091eb0 2020-03-19T21:18:20.063 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.052+0000 7fc35f7fe700 1 -- 172.21.15.170:0/2819985477 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1868349113 0 0) 0x7fc37010c4f0 con 0x7fc37011fab0 2020-03-19T21:18:20.063 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.052+0000 7fc376026700 1 --2- 172.21.15.170:0/2819985477 >> v2:172.21.15.170:3300/0 conn(0x7fc370091eb0 0x7fc370064ac0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7fc370065f80 tx=0x7fc360007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:20.063 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.052+0000 7fc376026700 1 --2- 172.21.15.170:0/2819985477 >> v2:172.21.15.170:3300/0 conn(0x7fc370091eb0 0x7fc370064ac0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7fc370065f80 tx=0x7fc360007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:20.064 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.052+0000 7fc376026700 1 --2- 172.21.15.170:0/2819985477 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc370091eb0 0x7fc370064ac0 secure :-1 s=READY pgs=165 cs=0 l=1 rx=0x7fc370065f80 tx=0x7fc360007a40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:20.064 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.052+0000 7fc376026700 1 --2- 172.21.15.170:0/2819985477 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc370091eb0 0x7fc370064ac0 secure :-1 s=THROTTLE_DONE pgs=165 cs=0 l=1 rx=0x7fc370065f80 tx=0x7fc360007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:20.064 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.052+0000 7fc35f7fe700 1 -- 172.21.15.170:0/2819985477 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fc360011e30 con 0x7fc370091eb0 2020-03-19T21:18:20.064 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.052+0000 7fc376026700 1 --2- 172.21.15.170:0/2819985477 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc370091eb0 0x7fc370064ac0 secure :-1 s=THROTTLE_DONE pgs=165 cs=0 l=1 rx=0x7fc370065f80 tx=0x7fc360007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:20.064 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.052+0000 7fc376026700 1 --2- 172.21.15.170:0/2819985477 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc370091eb0 0x7fc370064ac0 secure :-1 s=THROTTLE_DONE pgs=165 cs=0 l=1 rx=0x7fc370065f80 tx=0x7fc360007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:20.064 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.052+0000 7fc35f7fe700 1 -- 172.21.15.170:0/2819985477 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc360007150 con 0x7fc370091eb0 2020-03-19T21:18:20.064 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.052+0000 7fc35f7fe700 1 -- 172.21.15.170:0/2819985477 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fc3600073f0 con 0x7fc370091eb0 2020-03-19T21:18:20.065 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.052+0000 7fc37828a700 1 -- 172.21.15.170:0/2819985477 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc370066b40 con 0x7fc370091eb0 2020-03-19T21:18:20.065 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.053+0000 7fc37828a700 1 -- 172.21.15.170:0/2819985477 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc37010b370 con 0x7fc370091eb0 2020-03-19T21:18:20.065 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.053+0000 7fc376026700 1 --2- 172.21.15.170:0/2819985477 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc370091eb0 0x7fc370064ac0 secure :-1 s=THROTTLE_DONE pgs=165 cs=0 l=1 rx=0x7fc370065f80 tx=0x7fc360007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:20.065 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.053+0000 7fc35f7fe700 1 -- 172.21.15.170:0/2819985477 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 4) v1 ==== 29603+0+0 (secure 0 0 0) 0x7fc3600056c0 con 0x7fc370091eb0 2020-03-19T21:18:20.065 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.053+0000 7fc376026700 1 --2- 172.21.15.170:0/2819985477 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc370091eb0 0x7fc370064ac0 secure :-1 s=THROTTLE_DONE pgs=165 cs=0 l=1 rx=0x7fc370065f80 tx=0x7fc360007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:20.065 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.053+0000 7fc35f7fe700 1 --2- 172.21.15.170:0/2819985477 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7fc35002a410 0x7fc35002c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:20.065 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.053+0000 7fc35f7fe700 1 -- 172.21.15.170:0/2819985477 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3403+0+0 (secure 0 0 0) 0x7fc360034970 con 0x7fc370091eb0 2020-03-19T21:18:20.066 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.053+0000 7fc375825700 1 --2- 172.21.15.170:0/2819985477 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7fc35002a410 0x7fc35002c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:20.066 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.054+0000 7fc375825700 1 --2- 172.21.15.170:0/2819985477 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7fc35002a410 0x7fc35002c8d0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7fc36c000ee0 tx=0x7fc36c001670).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:20.066 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.054+0000 7fc375825700 1 --2- 172.21.15.170:0/2819985477 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7fc35002a410 0x7fc35002c8d0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7fc36c000ee0 tx=0x7fc36c001670).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:20.066 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.054+0000 7fc375825700 1 --2- 172.21.15.170:0/2819985477 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7fc35002a410 0x7fc35002c8d0 secure :-1 s=READY pgs=41 cs=0 l=1 rx=0x7fc36c000ee0 tx=0x7fc36c001670).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:20.066 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.054+0000 7fc37828a700 1 -- 172.21.15.170:0/2819985477 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc370056db0 con 0x7fc370091eb0 2020-03-19T21:18:20.067 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.057+0000 7fc376026700 1 --2- 172.21.15.170:0/2819985477 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc370091eb0 0x7fc370064ac0 secure :-1 s=THROTTLE_DONE pgs=165 cs=0 l=1 rx=0x7fc370065f80 tx=0x7fc360007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:20.067 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.057+0000 7fc35f7fe700 1 -- 172.21.15.170:0/2819985477 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+126772 (secure 0 0 0) 0x7fc360005950 con 0x7fc370091eb0 2020-03-19T21:18:20.197 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.195+0000 7fc37828a700 1 -- 172.21.15.170:0/2819985477 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fc3700693f0 con 0x7fc370091eb0 2020-03-19T21:18:20.197 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.195+0000 7fc376026700 1 --2- 172.21.15.170:0/2819985477 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc370091eb0 0x7fc370064ac0 secure :-1 s=THROTTLE_DONE pgs=165 cs=0 l=1 rx=0x7fc370065f80 tx=0x7fc360007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:20.198 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.195+0000 7fc35f7fe700 1 -- 172.21.15.170:0/2819985477 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc360006070 con 0x7fc370091eb0 2020-03-19T21:18:20.198 INFO:teuthology.orchestra.run.smithi170.stdout:34359738374 2020-03-19T21:18:20.199 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.196+0000 7fc37828a700 1 -- 172.21.15.170:0/2819985477 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7fc35002a410 msgr2=0x7fc35002c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:20.199 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.196+0000 7fc37828a700 1 --2- 172.21.15.170:0/2819985477 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7fc35002a410 0x7fc35002c8d0 secure :-1 s=READY pgs=41 cs=0 l=1 rx=0x7fc36c000ee0 tx=0x7fc36c001670).stop 2020-03-19T21:18:20.199 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.196+0000 7fc37828a700 1 -- 172.21.15.170:0/2819985477 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc370091eb0 msgr2=0x7fc370064ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:20.199 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.196+0000 7fc37828a700 1 --2- 172.21.15.170:0/2819985477 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc370091eb0 0x7fc370064ac0 secure :-1 s=READY pgs=165 cs=0 l=1 rx=0x7fc370065f80 tx=0x7fc360007a40).stop 2020-03-19T21:18:20.199 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.197+0000 7fc37828a700 1 -- 172.21.15.170:0/2819985477 shutdown_connections 2020-03-19T21:18:20.200 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.197+0000 7fc37828a700 1 --2- 172.21.15.170:0/2819985477 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7fc35002a410 0x7fc35002c8d0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:20.200 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.197+0000 7fc37828a700 1 --2- 172.21.15.170:0/2819985477 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc370091eb0 0x7fc370064ac0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:20.200 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.197+0000 7fc37828a700 1 -- 172.21.15.170:0/2819985477 >> 172.21.15.170:0/2819985477 conn(0x7fc370091390 msgr2=0x7fc370060110 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:20.200 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.197+0000 7fc37828a700 1 -- 172.21.15.170:0/2819985477 shutdown_connections 2020-03-19T21:18:20.200 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.197+0000 7fc37828a700 1 -- 172.21.15.170:0/2819985477 wait complete. 2020-03-19T21:18:20.209 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738374 got 34359738374 for osd.2 2020-03-19T21:18:20.209 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2020-03-19T21:18:20.209 INFO:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early pg dump --format=json 2020-03-19T21:18:20.317 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.312+0000 7f4a1f08e700 1 Processor -- start 2020-03-19T21:18:20.317 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.312+0000 7f4a1f08e700 1 -- start start 2020-03-19T21:18:20.317 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.313+0000 7f4a1f08e700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f4a180907c0 0x7f4a1808daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T21:18:20.317 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.313+0000 7f4a1f08e700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a180605e0 con 0x7f4a1808d000 2020-03-19T21:18:20.317 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.313+0000 7f4a1f08e700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a1808ef70 con 0x7f4a180907c0 2020-03-19T21:18:20.318 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.313+0000 7f4a17fff700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f4a180907c0 0x7f4a1808daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:20.318 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.313+0000 7f4a17fff700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f4a180907c0 0x7f4a1808daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53052/0 (socket says 172.21.15.170:53052) 2020-03-19T21:18:20.318 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.313+0000 7f4a1ce2a700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f4a1808d000 0x7f4a1808d3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:53814/0 (socket says 172.21.15.170:53814) 2020-03-19T21:18:20.318 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.313+0000 7f4a17fff700 1 -- 172.21.15.170:0/2729251240 learned_addr learned my addr 172.21.15.170:0/2729251240 (peer_addr_for_me v2:172.21.15.170:0/0) 2020-03-19T21:18:20.318 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.313+0000 7f4a17fff700 1 -- 172.21.15.170:0/2729251240 >> v1:172.21.15.170:6789/0 conn(0x7f4a1808d000 legacy=0x7f4a1808d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:20.318 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.313+0000 7f4a17fff700 1 -- 172.21.15.170:0/2729251240 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a18060810 con 0x7f4a180907c0 2020-03-19T21:18:20.318 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.313+0000 7f4a177fe700 1 -- 172.21.15.170:0/2729251240 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2003227819 0 0) 0x7f4a180605e0 con 0x7f4a1808d000 2020-03-19T21:18:20.319 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.313+0000 7f4a17fff700 1 --2- 172.21.15.170:0/2729251240 >> v2:172.21.15.170:3300/0 conn(0x7f4a180907c0 0x7f4a1808daf0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f4a18090c70 tx=0x7f4a0c0088e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:20.319 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.313+0000 7f4a17fff700 1 --2- 172.21.15.170:0/2729251240 >> v2:172.21.15.170:3300/0 conn(0x7f4a180907c0 0x7f4a1808daf0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f4a18090c70 tx=0x7f4a0c0088e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:20.319 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.313+0000 7f4a17fff700 1 --2- 172.21.15.170:0/2729251240 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a180907c0 0x7f4a1808daf0 secure :-1 s=READY pgs=167 cs=0 l=1 rx=0x7f4a18090c70 tx=0x7f4a0c0088e0).ready entity=mon.0 client_cookie=e1a67a1b6a9ed666 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:20.319 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.313+0000 7f4a17fff700 1 --2- 172.21.15.170:0/2729251240 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a180907c0 0x7f4a1808daf0 secure :-1 s=THROTTLE_DONE pgs=167 cs=0 l=1 rx=0x7f4a18090c70 tx=0x7f4a0c0088e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:20.319 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.313+0000 7f4a177fe700 1 -- 172.21.15.170:0/2729251240 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f4a0c0114b0 con 0x7f4a180907c0 2020-03-19T21:18:20.319 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.313+0000 7f4a17fff700 1 --2- 172.21.15.170:0/2729251240 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a180907c0 0x7f4a1808daf0 secure :-1 s=THROTTLE_DONE pgs=167 cs=0 l=1 rx=0x7f4a18090c70 tx=0x7f4a0c0088e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:20.319 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.313+0000 7f4a17fff700 1 --2- 172.21.15.170:0/2729251240 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a180907c0 0x7f4a1808daf0 secure :-1 s=THROTTLE_DONE pgs=167 cs=0 l=1 rx=0x7f4a18090c70 tx=0x7f4a0c0088e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:20.320 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.313+0000 7f4a177fe700 1 -- 172.21.15.170:0/2729251240 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a0c011610 con 0x7f4a180907c0 2020-03-19T21:18:20.320 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.313+0000 7f4a177fe700 1 -- 172.21.15.170:0/2729251240 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f4a0c0118b0 con 0x7f4a180907c0 2020-03-19T21:18:20.320 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.313+0000 7f4a1f08e700 1 -- 172.21.15.170:0/2729251240 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a180907c0 msgr2=0x7f4a1808daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:20.320 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.313+0000 7f4a1f08e700 1 --2- 172.21.15.170:0/2729251240 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a180907c0 0x7f4a1808daf0 secure :-1 s=READY pgs=167 cs=0 l=1 rx=0x7f4a18090c70 tx=0x7f4a0c0088e0).stop 2020-03-19T21:18:20.320 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.314+0000 7f4a1f08e700 1 -- 172.21.15.170:0/2729251240 shutdown_connections 2020-03-19T21:18:20.320 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.314+0000 7f4a1f08e700 1 --2- 172.21.15.170:0/2729251240 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a180907c0 0x7f4a1808daf0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:20.320 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.314+0000 7f4a1f08e700 1 -- 172.21.15.170:0/2729251240 >> 172.21.15.170:0/2729251240 conn(0x7f4a18091540 msgr2=0x7f4a18091950 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:20.321 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.314+0000 7f4a1f08e700 1 -- 172.21.15.170:0/2729251240 shutdown_connections 2020-03-19T21:18:20.321 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.314+0000 7f4a1f08e700 1 -- 172.21.15.170:0/2729251240 wait complete. 2020-03-19T21:18:20.321 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.315+0000 7f4a1f08e700 1 Processor -- start 2020-03-19T21:18:20.321 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.315+0000 7f4a1f08e700 1 -- start start 2020-03-19T21:18:20.324 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.315+0000 7f4a1f08e700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f4a1808d000 0x7f4a18122260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:20.325 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.315+0000 7f4a1f08e700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a18121050 con 0x7f4a180907c0 2020-03-19T21:18:20.325 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.315+0000 7f4a1f08e700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a1811edf0 con 0x7f4a1808d000 2020-03-19T21:18:20.325 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.315+0000 7f4a1ce2a700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f4a1808d000 0x7f4a18122260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:20.325 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.315+0000 7f4a17fff700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f4a180907c0 0x7f4a181226c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:53820/0 (socket says 172.21.15.170:53820) 2020-03-19T21:18:20.325 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.315+0000 7f4a1ce2a700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f4a1808d000 0x7f4a18122260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53054/0 (socket says 172.21.15.170:53054) 2020-03-19T21:18:20.325 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.315+0000 7f4a17fff700 1 -- 172.21.15.170:0/2233776664 learned_addr learned my addr 172.21.15.170:0/2233776664 (peer_addr_for_me v1:172.21.15.170:0/0) 2020-03-19T21:18:20.325 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.316+0000 7f4a1ce2a700 1 -- 172.21.15.170:0/2233776664 >> v1:172.21.15.170:6789/0 conn(0x7f4a180907c0 legacy=0x7f4a181226c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:20.326 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.316+0000 7f4a1ce2a700 1 -- 172.21.15.170:0/2233776664 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a18120d40 con 0x7f4a1808d000 2020-03-19T21:18:20.326 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.316+0000 7f4a15ffb700 1 -- 172.21.15.170:0/2233776664 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2250426133 0 0) 0x7f4a18121050 con 0x7f4a180907c0 2020-03-19T21:18:20.326 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.316+0000 7f4a1ce2a700 1 --2- 172.21.15.170:0/2233776664 >> v2:172.21.15.170:3300/0 conn(0x7f4a1808d000 0x7f4a18122260 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f4a18120090 tx=0x7f4a080108f0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:20.326 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.316+0000 7f4a1ce2a700 1 --2- 172.21.15.170:0/2233776664 >> v2:172.21.15.170:3300/0 conn(0x7f4a1808d000 0x7f4a18122260 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f4a18120090 tx=0x7f4a080108f0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:20.326 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.316+0000 7f4a1ce2a700 1 --2- 172.21.15.170:0/2233776664 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a1808d000 0x7f4a18122260 secure :-1 s=READY pgs=169 cs=0 l=1 rx=0x7f4a18120090 tx=0x7f4a080108f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:20.326 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.316+0000 7f4a1ce2a700 1 --2- 172.21.15.170:0/2233776664 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a1808d000 0x7f4a18122260 secure :-1 s=THROTTLE_DONE pgs=169 cs=0 l=1 rx=0x7f4a18120090 tx=0x7f4a080108f0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:20.326 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.316+0000 7f4a15ffb700 1 -- 172.21.15.170:0/2233776664 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f4a08011440 con 0x7f4a1808d000 2020-03-19T21:18:20.327 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.316+0000 7f4a1ce2a700 1 --2- 172.21.15.170:0/2233776664 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a1808d000 0x7f4a18122260 secure :-1 s=THROTTLE_DONE pgs=169 cs=0 l=1 rx=0x7f4a18120090 tx=0x7f4a080108f0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:20.327 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.316+0000 7f4a1ce2a700 1 --2- 172.21.15.170:0/2233776664 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a1808d000 0x7f4a18122260 secure :-1 s=THROTTLE_DONE pgs=169 cs=0 l=1 rx=0x7f4a18120090 tx=0x7f4a080108f0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:20.327 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.316+0000 7f4a15ffb700 1 -- 172.21.15.170:0/2233776664 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a080115a0 con 0x7f4a1808d000 2020-03-19T21:18:20.327 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.316+0000 7f4a15ffb700 1 -- 172.21.15.170:0/2233776664 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f4a08011900 con 0x7f4a1808d000 2020-03-19T21:18:20.327 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.316+0000 7f4a1f08e700 1 -- 172.21.15.170:0/2233776664 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a1808cb10 con 0x7f4a1808d000 2020-03-19T21:18:20.327 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.316+0000 7f4a1f08e700 1 -- 172.21.15.170:0/2233776664 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a1811f240 con 0x7f4a1808d000 2020-03-19T21:18:20.327 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.316+0000 7f4a1ce2a700 1 --2- 172.21.15.170:0/2233776664 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a1808d000 0x7f4a18122260 secure :-1 s=THROTTLE_DONE pgs=169 cs=0 l=1 rx=0x7f4a18120090 tx=0x7f4a080108f0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:20.328 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.317+0000 7f4a15ffb700 1 -- 172.21.15.170:0/2233776664 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 4) v1 ==== 29603+0+0 (secure 0 0 0) 0x7f4a0800b0a0 con 0x7f4a1808d000 2020-03-19T21:18:20.328 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.317+0000 7f4a1ce2a700 1 --2- 172.21.15.170:0/2233776664 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a1808d000 0x7f4a18122260 secure :-1 s=THROTTLE_DONE pgs=169 cs=0 l=1 rx=0x7f4a18120090 tx=0x7f4a080108f0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:20.328 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.317+0000 7f4a15ffb700 1 --2- 172.21.15.170:0/2233776664 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f49f80281b0 0x7f49f802a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:20.328 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.317+0000 7f4a15ffb700 1 -- 172.21.15.170:0/2233776664 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3403+0+0 (secure 0 0 0) 0x7f4a08034020 con 0x7f4a1808d000 2020-03-19T21:18:20.328 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.317+0000 7f4a17fff700 1 --2- 172.21.15.170:0/2233776664 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f49f80281b0 0x7f49f802a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:20.328 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.317+0000 7f4a1f08e700 1 -- 172.21.15.170:0/2233776664 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a181207e0 con 0x7f4a1808d000 2020-03-19T21:18:20.329 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.317+0000 7f4a17fff700 1 --2- 172.21.15.170:0/2233776664 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f49f80281b0 0x7f49f802a670 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f4a0c010280 tx=0x7f4a18067850).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:20.329 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.317+0000 7f4a17fff700 1 --2- 172.21.15.170:0/2233776664 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f49f80281b0 0x7f49f802a670 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f4a0c010280 tx=0x7f4a18067850).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:20.329 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.317+0000 7f4a17fff700 1 --2- 172.21.15.170:0/2233776664 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f49f80281b0 0x7f49f802a670 secure :-1 s=READY pgs=42 cs=0 l=1 rx=0x7f4a0c010280 tx=0x7f4a18067850).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:20.329 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.321+0000 7f4a1ce2a700 1 --2- 172.21.15.170:0/2233776664 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a1808d000 0x7f4a18122260 secure :-1 s=THROTTLE_DONE pgs=169 cs=0 l=1 rx=0x7f4a18120090 tx=0x7f4a080108f0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:20.329 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.321+0000 7f4a15ffb700 1 -- 172.21.15.170:0/2233776664 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+126772 (secure 0 0 0) 0x7f4a08011d50 con 0x7f4a1808d000 2020-03-19T21:18:20.457 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.455+0000 7f4a1f08e700 1 -- 172.21.15.170:0/2233776664 --> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4a1808c320 con 0x7f49f80281b0 2020-03-19T21:18:20.460 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.457+0000 7f4a17fff700 1 --2- 172.21.15.170:0/2233776664 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f49f80281b0 0x7f49f802a670 secure :-1 s=THROTTLE_DONE pgs=42 cs=0 l=1 rx=0x7f4a0c010280 tx=0x7f4a18067850).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:20.460 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.457+0000 7f4a15ffb700 1 -- 172.21.15.170:0/2233776664 <== mgr.4102 v2:172.21.15.170:6824/20154 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+63967 (secure 0 0 0) 0x7f4a1808c320 con 0x7f49f80281b0 2020-03-19T21:18:20.460 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:18:20.462 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.458+0000 7f4a1f08e700 1 -- 172.21.15.170:0/2233776664 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f49f80281b0 msgr2=0x7f49f802a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:20.462 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.458+0000 7f4a1f08e700 1 --2- 172.21.15.170:0/2233776664 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f49f80281b0 0x7f49f802a670 secure :-1 s=READY pgs=42 cs=0 l=1 rx=0x7f4a0c010280 tx=0x7f4a18067850).stop 2020-03-19T21:18:20.462 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.458+0000 7f4a1f08e700 1 -- 172.21.15.170:0/2233776664 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a1808d000 msgr2=0x7f4a18122260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:20.462 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.458+0000 7f4a1f08e700 1 --2- 172.21.15.170:0/2233776664 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a1808d000 0x7f4a18122260 secure :-1 s=READY pgs=169 cs=0 l=1 rx=0x7f4a18120090 tx=0x7f4a080108f0).stop 2020-03-19T21:18:20.462 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.459+0000 7f4a1f08e700 1 -- 172.21.15.170:0/2233776664 shutdown_connections 2020-03-19T21:18:20.462 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.459+0000 7f4a1f08e700 1 --2- 172.21.15.170:0/2233776664 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f49f80281b0 0x7f49f802a670 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:20.463 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.459+0000 7f4a1f08e700 1 --2- 172.21.15.170:0/2233776664 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a1808d000 0x7f4a18122260 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:20.463 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.459+0000 7f4a1f08e700 1 -- 172.21.15.170:0/2233776664 >> 172.21.15.170:0/2233776664 conn(0x7f4a18091540 msgr2=0x7f4a180629b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:20.463 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.459+0000 7f4a1f08e700 1 -- 172.21.15.170:0/2233776664 shutdown_connections 2020-03-19T21:18:20.463 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.459+0000 7f4a1f08e700 1 -- 172.21.15.170:0/2233776664 wait complete. 2020-03-19T21:18:20.463 INFO:teuthology.orchestra.run.smithi170.stderr:dumped all 2020-03-19T21:18:20.471 INFO:teuthology.orchestra.run.smithi170.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2020-03-19T21:18:18.509800+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2286,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":314572800,"kb_used":3146224,"kb_used_data":472,"kb_used_omap":0,"kb_used_meta":3145728,"kb_avail":311426576,"statfs":{"total":322122547200,"available":318900813824,"internally_reserved":3221225472,"allocated":483328,"data_stored":132129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":3221225472},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"3.7","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2020-03-19T21:18:13.525855+0000","last_change":"2020-03-19T21:18:13.525398+0000","last_active":"2020-03-19T21:18:13.525855+0000","last_peered":"2020-03-19T21:18:13.525855+0000","last_clean":"2020-03-19T21:18:13.525855+0000","last_became_active":"2020-03-19T21:18:13.525041+0000","last_became_peered":"2020-03-19T21:18:13.525041+0000","last_unstale":"2020-03-19T21:18:13.525855+0000","last_undegraded":"2020-03-19T21:18:13.525855+0000","last_fullsized":"2020-03-19T21:18:13.525855+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:13.185694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:13.185694+0000","last_clean_scrub_stamp":"2020-03-19T21:18:13.185694+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":"13","reported_epoch":"15","state":"active+clean","last_fresh":"2020-03-19T21:18:13.521281+0000","last_change":"2020-03-19T21:18:10.186965+0000","last_active":"2020-03-19T21:18:13.521281+0000","last_peered":"2020-03-19T21:18:13.521281+0000","last_clean":"2020-03-19T21:18:13.521281+0000","last_became_active":"2020-03-19T21:18:10.186584+0000","last_became_peered":"2020-03-19T21:18:10.186584+0000","last_unstale":"2020-03-19T21:18:13.521281+0000","last_undegraded":"2020-03-19T21:18:13.521281+0000","last_fullsized":"2020-03-19T21:18:13.521281+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:09.169195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:09.169195+0000","last_clean_scrub_stamp":"2020-03-19T21:18:09.169195+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"16'3","reported_seq":"13","reported_epoch":"16","state":"active+clean","last_fresh":"2020-03-19T21:18:16.214216+0000","last_change":"2020-03-19T21:18:13.197989+0000","last_active":"2020-03-19T21:18:16.214216+0000","last_peered":"2020-03-19T21:18:16.214216+0000","last_clean":"2020-03-19T21:18:16.214216+0000","last_became_active":"2020-03-19T21:18:13.197671+0000","last_became_peered":"2020-03-19T21:18:13.197671+0000","last_unstale":"2020-03-19T21:18:16.214216+0000","last_undegraded":"2020-03-19T21:18:16.214216+0000","last_fullsized":"2020-03-19T21:18:16.214216+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:12.182524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:12.182524+0000","last_clean_scrub_stamp":"2020-03-19T21:18:12.182524+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2020-03-19T21:18:13.526546+0000","last_change":"2020-03-19T21:18:13.526027+0000","last_active":"2020-03-19T21:18:13.526546+0000","last_peered":"2020-03-19T21:18:13.526546+0000","last_clean":"2020-03-19T21:18:13.526546+0000","last_became_active":"2020-03-19T21:18:13.524563+0000","last_became_peered":"2020-03-19T21:18:13.524563+0000","last_unstale":"2020-03-19T21:18:13.526546+0000","last_undegraded":"2020-03-19T21:18:13.526546+0000","last_fullsized":"2020-03-19T21:18:13.526546+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:13.185694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:13.185694+0000","last_clean_scrub_stamp":"2020-03-19T21:18:13.185694+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":"12","reported_epoch":"15","state":"active+clean","last_fresh":"2020-03-19T21:18:13.524035+0000","last_change":"2020-03-19T21:18:10.184204+0000","last_active":"2020-03-19T21:18:13.524035+0000","last_peered":"2020-03-19T21:18:13.524035+0000","last_clean":"2020-03-19T21:18:13.524035+0000","last_became_active":"2020-03-19T21:18:10.183810+0000","last_became_peered":"2020-03-19T21:18:10.183810+0000","last_unstale":"2020-03-19T21:18:13.524035+0000","last_undegraded":"2020-03-19T21:18:13.524035+0000","last_fullsized":"2020-03-19T21:18:13.524035+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:09.169195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:09.169195+0000","last_clean_scrub_stamp":"2020-03-19T21:18:09.169195+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"16'7","reported_seq":"17","reported_epoch":"16","state":"active+clean","last_fresh":"2020-03-19T21:18:16.216574+0000","last_change":"2020-03-19T21:18:13.196212+0000","last_active":"2020-03-19T21:18:16.216574+0000","last_peered":"2020-03-19T21:18:16.216574+0000","last_clean":"2020-03-19T21:18:16.216574+0000","last_became_active":"2020-03-19T21:18:13.195896+0000","last_became_peered":"2020-03-19T21:18:13.195896+0000","last_unstale":"2020-03-19T21:18:16.216574+0000","last_undegraded":"2020-03-19T21:18:16.216574+0000","last_fullsized":"2020-03-19T21:18:16.216574+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:12.182524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:12.182524+0000","last_clean_scrub_stamp":"2020-03-19T21:18:12.182524+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":566,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2020-03-19T21:18:13.524356+0000","last_change":"2020-03-19T21:18:13.524166+0000","last_active":"2020-03-19T21:18:13.524356+0000","last_peered":"2020-03-19T21:18:13.524356+0000","last_clean":"2020-03-19T21:18:13.524356+0000","last_became_active":"2020-03-19T21:18:13.523806+0000","last_became_peered":"2020-03-19T21:18:13.523806+0000","last_unstale":"2020-03-19T21:18:13.524356+0000","last_undegraded":"2020-03-19T21:18:13.524356+0000","last_fullsized":"2020-03-19T21:18:13.524356+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:13.185694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:13.185694+0000","last_clean_scrub_stamp":"2020-03-19T21:18:13.185694+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":"12","reported_epoch":"15","state":"active+clean","last_fresh":"2020-03-19T21:18:13.524452+0000","last_change":"2020-03-19T21:18:10.184210+0000","last_active":"2020-03-19T21:18:13.524452+0000","last_peered":"2020-03-19T21:18:13.524452+0000","last_clean":"2020-03-19T21:18:13.524452+0000","last_became_active":"2020-03-19T21:18:10.183788+0000","last_became_peered":"2020-03-19T21:18:10.183788+0000","last_unstale":"2020-03-19T21:18:13.524452+0000","last_undegraded":"2020-03-19T21:18:13.524452+0000","last_fullsized":"2020-03-19T21:18:13.524452+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:09.169195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:09.169195+0000","last_clean_scrub_stamp":"2020-03-19T21:18:09.169195+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"16'8","reported_seq":"18","reported_epoch":"16","state":"active+clean","last_fresh":"2020-03-19T21:18:16.216506+0000","last_change":"2020-03-19T21:18:13.197291+0000","last_active":"2020-03-19T21:18:16.216506+0000","last_peered":"2020-03-19T21:18:16.216506+0000","last_clean":"2020-03-19T21:18:16.216506+0000","last_became_active":"2020-03-19T21:18:13.196823+0000","last_became_peered":"2020-03-19T21:18:13.196823+0000","last_unstale":"2020-03-19T21:18:16.216506+0000","last_undegraded":"2020-03-19T21:18:16.216506+0000","last_fullsized":"2020-03-19T21:18:16.216506+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:12.182524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:12.182524+0000","last_clean_scrub_stamp":"2020-03-19T21:18:12.182524+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2020-03-19T21:18:13.528411+0000","last_change":"2020-03-19T21:18:13.528239+0000","last_active":"2020-03-19T21:18:13.528411+0000","last_peered":"2020-03-19T21:18:13.528411+0000","last_clean":"2020-03-19T21:18:13.528411+0000","last_became_active":"2020-03-19T21:18:13.527916+0000","last_became_peered":"2020-03-19T21:18:13.527916+0000","last_unstale":"2020-03-19T21:18:13.528411+0000","last_undegraded":"2020-03-19T21:18:13.528411+0000","last_fullsized":"2020-03-19T21:18:13.528411+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:13.185694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:13.185694+0000","last_clean_scrub_stamp":"2020-03-19T21:18:13.185694+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":"12","reported_epoch":"15","state":"active+clean","last_fresh":"2020-03-19T21:18:13.519207+0000","last_change":"2020-03-19T21:18:10.187875+0000","last_active":"2020-03-19T21:18:13.519207+0000","last_peered":"2020-03-19T21:18:13.519207+0000","last_clean":"2020-03-19T21:18:13.519207+0000","last_became_active":"2020-03-19T21:18:10.187570+0000","last_became_peered":"2020-03-19T21:18:10.187570+0000","last_unstale":"2020-03-19T21:18:13.519207+0000","last_undegraded":"2020-03-19T21:18:13.519207+0000","last_fullsized":"2020-03-19T21:18:13.519207+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:09.169195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:09.169195+0000","last_clean_scrub_stamp":"2020-03-19T21:18:09.169195+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2020-03-19T21:18:13.521933+0000","last_change":"2020-03-19T21:18:13.197535+0000","last_active":"2020-03-19T21:18:13.521933+0000","last_peered":"2020-03-19T21:18:13.521933+0000","last_clean":"2020-03-19T21:18:13.521933+0000","last_became_active":"2020-03-19T21:18:13.197224+0000","last_became_peered":"2020-03-19T21:18:13.197224+0000","last_unstale":"2020-03-19T21:18:13.521933+0000","last_undegraded":"2020-03-19T21:18:13.521933+0000","last_fullsized":"2020-03-19T21:18:13.521933+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:12.182524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:12.182524+0000","last_clean_scrub_stamp":"2020-03-19T21:18:12.182524+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2020-03-19T21:18:13.524858+0000","last_change":"2020-03-19T21:18:13.524218+0000","last_active":"2020-03-19T21:18:13.524858+0000","last_peered":"2020-03-19T21:18:13.524858+0000","last_clean":"2020-03-19T21:18:13.524858+0000","last_became_active":"2020-03-19T21:18:13.523902+0000","last_became_peered":"2020-03-19T21:18:13.523902+0000","last_unstale":"2020-03-19T21:18:13.524858+0000","last_undegraded":"2020-03-19T21:18:13.524858+0000","last_fullsized":"2020-03-19T21:18:13.524858+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:13.185694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:13.185694+0000","last_clean_scrub_stamp":"2020-03-19T21:18:13.185694+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":"12","reported_epoch":"15","state":"active+clean","last_fresh":"2020-03-19T21:18:13.523332+0000","last_change":"2020-03-19T21:18:10.184801+0000","last_active":"2020-03-19T21:18:13.523332+0000","last_peered":"2020-03-19T21:18:13.523332+0000","last_clean":"2020-03-19T21:18:13.523332+0000","last_became_active":"2020-03-19T21:18:10.184433+0000","last_became_peered":"2020-03-19T21:18:10.184433+0000","last_unstale":"2020-03-19T21:18:13.523332+0000","last_undegraded":"2020-03-19T21:18:13.523332+0000","last_fullsized":"2020-03-19T21:18:13.523332+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:09.169195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:09.169195+0000","last_clean_scrub_stamp":"2020-03-19T21:18:09.169195+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"16'5","reported_seq":"15","reported_epoch":"16","state":"active+clean","last_fresh":"2020-03-19T21:18:16.213106+0000","last_change":"2020-03-19T21:18:13.195202+0000","last_active":"2020-03-19T21:18:16.213106+0000","last_peered":"2020-03-19T21:18:16.213106+0000","last_clean":"2020-03-19T21:18:16.213106+0000","last_became_active":"2020-03-19T21:18:13.194849+0000","last_became_peered":"2020-03-19T21:18:13.194849+0000","last_unstale":"2020-03-19T21:18:16.213106+0000","last_undegraded":"2020-03-19T21:18:16.213106+0000","last_fullsized":"2020-03-19T21:18:16.213106+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:12.182524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:12.182524+0000","last_clean_scrub_stamp":"2020-03-19T21:18:12.182524+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2020-03-19T21:18:13.523537+0000","last_change":"2020-03-19T21:18:13.523284+0000","last_active":"2020-03-19T21:18:13.523537+0000","last_peered":"2020-03-19T21:18:13.523537+0000","last_clean":"2020-03-19T21:18:13.523537+0000","last_became_active":"2020-03-19T21:18:13.522937+0000","last_became_peered":"2020-03-19T21:18:13.522937+0000","last_unstale":"2020-03-19T21:18:13.523537+0000","last_undegraded":"2020-03-19T21:18:13.523537+0000","last_fullsized":"2020-03-19T21:18:13.523537+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:13.185694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:13.185694+0000","last_clean_scrub_stamp":"2020-03-19T21:18:13.185694+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":"13","reported_epoch":"15","state":"active+clean","last_fresh":"2020-03-19T21:18:13.521872+0000","last_change":"2020-03-19T21:18:10.183717+0000","last_active":"2020-03-19T21:18:13.521872+0000","last_peered":"2020-03-19T21:18:13.521872+0000","last_clean":"2020-03-19T21:18:13.521872+0000","last_became_active":"2020-03-19T21:18:10.183404+0000","last_became_peered":"2020-03-19T21:18:10.183404+0000","last_unstale":"2020-03-19T21:18:13.521872+0000","last_undegraded":"2020-03-19T21:18:13.521872+0000","last_fullsized":"2020-03-19T21:18:13.521872+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:09.169195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:09.169195+0000","last_clean_scrub_stamp":"2020-03-19T21:18:09.169195+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"16'1","reported_seq":"11","reported_epoch":"16","state":"active+clean","last_fresh":"2020-03-19T21:18:14.518247+0000","last_change":"2020-03-19T21:18:13.198955+0000","last_active":"2020-03-19T21:18:14.518247+0000","last_peered":"2020-03-19T21:18:14.518247+0000","last_clean":"2020-03-19T21:18:14.518247+0000","last_became_active":"2020-03-19T21:18:13.198498+0000","last_became_peered":"2020-03-19T21:18:13.198498+0000","last_unstale":"2020-03-19T21:18:14.518247+0000","last_undegraded":"2020-03-19T21:18:14.518247+0000","last_fullsized":"2020-03-19T21:18:14.518247+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:12.182524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:12.182524+0000","last_clean_scrub_stamp":"2020-03-19T21:18:12.182524+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2020-03-19T21:18:13.526493+0000","last_change":"2020-03-19T21:18:13.524754+0000","last_active":"2020-03-19T21:18:13.526493+0000","last_peered":"2020-03-19T21:18:13.526493+0000","last_clean":"2020-03-19T21:18:13.526493+0000","last_became_active":"2020-03-19T21:18:13.524437+0000","last_became_peered":"2020-03-19T21:18:13.524437+0000","last_unstale":"2020-03-19T21:18:13.526493+0000","last_undegraded":"2020-03-19T21:18:13.526493+0000","last_fullsized":"2020-03-19T21:18:13.526493+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:13.185694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:13.185694+0000","last_clean_scrub_stamp":"2020-03-19T21:18:13.185694+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":"12","reported_epoch":"15","state":"active+clean","last_fresh":"2020-03-19T21:18:13.524572+0000","last_change":"2020-03-19T21:18:10.184911+0000","last_active":"2020-03-19T21:18:13.524572+0000","last_peered":"2020-03-19T21:18:13.524572+0000","last_clean":"2020-03-19T21:18:13.524572+0000","last_became_active":"2020-03-19T21:18:10.184549+0000","last_became_peered":"2020-03-19T21:18:10.184549+0000","last_unstale":"2020-03-19T21:18:13.524572+0000","last_undegraded":"2020-03-19T21:18:13.524572+0000","last_fullsized":"2020-03-19T21:18:13.524572+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:09.169195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:09.169195+0000","last_clean_scrub_stamp":"2020-03-19T21:18:09.169195+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"16'6","reported_seq":"16","reported_epoch":"16","state":"active+clean","last_fresh":"2020-03-19T21:18:16.216619+0000","last_change":"2020-03-19T21:18:13.196469+0000","last_active":"2020-03-19T21:18:16.216619+0000","last_peered":"2020-03-19T21:18:16.216619+0000","last_clean":"2020-03-19T21:18:16.216619+0000","last_became_active":"2020-03-19T21:18:13.195947+0000","last_became_peered":"2020-03-19T21:18:13.195947+0000","last_unstale":"2020-03-19T21:18:16.216619+0000","last_undegraded":"2020-03-19T21:18:16.216619+0000","last_fullsized":"2020-03-19T21:18:16.216619+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:12.182524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:12.182524+0000","last_clean_scrub_stamp":"2020-03-19T21:18:12.182524+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":160,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2020-03-19T21:18:13.525722+0000","last_change":"2020-03-19T21:18:13.525236+0000","last_active":"2020-03-19T21:18:13.525722+0000","last_peered":"2020-03-19T21:18:13.525722+0000","last_clean":"2020-03-19T21:18:13.525722+0000","last_became_active":"2020-03-19T21:18:13.524938+0000","last_became_peered":"2020-03-19T21:18:13.524938+0000","last_unstale":"2020-03-19T21:18:13.525722+0000","last_undegraded":"2020-03-19T21:18:13.525722+0000","last_fullsized":"2020-03-19T21:18:13.525722+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:13.185694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:13.185694+0000","last_clean_scrub_stamp":"2020-03-19T21:18:13.185694+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":"12","reported_epoch":"15","state":"active+clean","last_fresh":"2020-03-19T21:18:13.523815+0000","last_change":"2020-03-19T21:18:10.182075+0000","last_active":"2020-03-19T21:18:13.523815+0000","last_peered":"2020-03-19T21:18:13.523815+0000","last_clean":"2020-03-19T21:18:13.523815+0000","last_became_active":"2020-03-19T21:18:10.181738+0000","last_became_peered":"2020-03-19T21:18:10.181738+0000","last_unstale":"2020-03-19T21:18:13.523815+0000","last_undegraded":"2020-03-19T21:18:13.523815+0000","last_fullsized":"2020-03-19T21:18:13.523815+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:09.169195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:09.169195+0000","last_clean_scrub_stamp":"2020-03-19T21:18:09.169195+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"16'8","reported_seq":"18","reported_epoch":"16","state":"active+clean","last_fresh":"2020-03-19T21:18:16.217013+0000","last_change":"2020-03-19T21:18:13.197376+0000","last_active":"2020-03-19T21:18:16.217013+0000","last_peered":"2020-03-19T21:18:16.217013+0000","last_clean":"2020-03-19T21:18:16.217013+0000","last_became_active":"2020-03-19T21:18:13.196089+0000","last_became_peered":"2020-03-19T21:18:13.196089+0000","last_unstale":"2020-03-19T21:18:16.217013+0000","last_undegraded":"2020-03-19T21:18:16.217013+0000","last_fullsized":"2020-03-19T21:18:16.217013+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:12.182524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:12.182524+0000","last_clean_scrub_stamp":"2020-03-19T21:18:12.182524+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1526,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":2286,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738374,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":1048724,"kb_used_data":140,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":103808876,"statfs":{"total":107374182400,"available":106300289024,"internally_reserved":1073741824,"allocated":143360,"data_stored":42553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47399999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61499999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738373,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":1048748,"kb_used_data":164,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":103808852,"statfs":{"total":107374182400,"available":106300264448,"internally_reserved":1073741824,"allocated":167936,"data_stored":44771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78400000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.8149999999999999}]}]},{"osd":1,"up_from":8,"seq":34359738373,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":1048752,"kb_used_data":168,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":103808848,"statfs":{"total":107374182400,"available":106300260352,"internally_reserved":1073741824,"allocated":172032,"data_stored":44805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.432}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58299999999999996}]}]}]}} 2020-03-19T21:18:20.474 INFO:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early pg dump --format=json 2020-03-19T21:18:20.586 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.577+0000 7f00da007700 1 Processor -- start 2020-03-19T21:18:20.586 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.577+0000 7f00da007700 1 -- start start 2020-03-19T21:18:20.587 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.579+0000 7f00da007700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f00d4066760 0x7f00d4066b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T21:18:20.587 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.579+0000 7f00da007700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00d40912a0 con 0x7f00d4090890 2020-03-19T21:18:20.587 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.579+0000 7f00da007700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00d408ddb0 con 0x7f00d4066760 2020-03-19T21:18:20.587 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.579+0000 7f00d37fe700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f00d4066760 0x7f00d4066b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:20.587 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.579+0000 7f00d2ffd700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f00d4090890 0x7f00d408bef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:53826/0 (socket says 172.21.15.170:53826) 2020-03-19T21:18:20.587 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.579+0000 7f00d2ffd700 1 -- 172.21.15.170:0/1450447090 learned_addr learned my addr 172.21.15.170:0/1450447090 (peer_addr_for_me v1:172.21.15.170:0/0) 2020-03-19T21:18:20.587 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.579+0000 7f00d37fe700 1 -- 172.21.15.170:0/1450447090 >> v1:172.21.15.170:6789/0 conn(0x7f00d4090890 legacy=0x7f00d408bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:20.588 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.579+0000 7f00d37fe700 1 -- 172.21.15.170:0/1450447090 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00d408d1b0 con 0x7f00d4066760 2020-03-19T21:18:20.588 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.579+0000 7f00d27fc700 1 -- 172.21.15.170:0/1450447090 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3395469250 0 0) 0x7f00d40912a0 con 0x7f00d4090890 2020-03-19T21:18:20.588 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.579+0000 7f00d37fe700 1 --2- 172.21.15.170:0/1450447090 >> v2:172.21.15.170:3300/0 conn(0x7f00d4066760 0x7f00d4066b40 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f00c4002f30 tx=0x7f00c4007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:20.588 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.579+0000 7f00d37fe700 1 --2- 172.21.15.170:0/1450447090 >> v2:172.21.15.170:3300/0 conn(0x7f00d4066760 0x7f00d4066b40 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f00c4002f30 tx=0x7f00c4007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:20.588 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.579+0000 7f00d37fe700 1 --2- 172.21.15.170:0/1450447090 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f00d4066760 0x7f00d4066b40 secure :-1 s=READY pgs=171 cs=0 l=1 rx=0x7f00c4002f30 tx=0x7f00c4007a40).ready entity=mon.0 client_cookie=20350041ee743bc4 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:20.588 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.579+0000 7f00d37fe700 1 --2- 172.21.15.170:0/1450447090 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f00d4066760 0x7f00d4066b40 secure :-1 s=THROTTLE_DONE pgs=171 cs=0 l=1 rx=0x7f00c4002f30 tx=0x7f00c4007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:20.589 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.579+0000 7f00d27fc700 1 -- 172.21.15.170:0/1450447090 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f00c4008e80 con 0x7f00d4066760 2020-03-19T21:18:20.589 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.579+0000 7f00d37fe700 1 --2- 172.21.15.170:0/1450447090 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f00d4066760 0x7f00d4066b40 secure :-1 s=THROTTLE_DONE pgs=171 cs=0 l=1 rx=0x7f00c4002f30 tx=0x7f00c4007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:20.589 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.579+0000 7f00d37fe700 1 --2- 172.21.15.170:0/1450447090 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f00d4066760 0x7f00d4066b40 secure :-1 s=THROTTLE_DONE pgs=171 cs=0 l=1 rx=0x7f00c4002f30 tx=0x7f00c4007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:20.589 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.579+0000 7f00d27fc700 1 -- 172.21.15.170:0/1450447090 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00c40113f0 con 0x7f00d4066760 2020-03-19T21:18:20.589 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.579+0000 7f00d27fc700 1 -- 172.21.15.170:0/1450447090 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f00c40116d0 con 0x7f00d4066760 2020-03-19T21:18:20.589 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.579+0000 7f00da007700 1 -- 172.21.15.170:0/1450447090 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f00d4066760 msgr2=0x7f00d4066b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:20.590 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.579+0000 7f00da007700 1 --2- 172.21.15.170:0/1450447090 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f00d4066760 0x7f00d4066b40 secure :-1 s=READY pgs=171 cs=0 l=1 rx=0x7f00c4002f30 tx=0x7f00c4007a40).stop 2020-03-19T21:18:20.590 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.579+0000 7f00da007700 1 -- 172.21.15.170:0/1450447090 shutdown_connections 2020-03-19T21:18:20.590 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.579+0000 7f00da007700 1 --2- 172.21.15.170:0/1450447090 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f00d4066760 0x7f00d4066b40 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:20.590 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.579+0000 7f00da007700 1 -- 172.21.15.170:0/1450447090 >> 172.21.15.170:0/1450447090 conn(0x7f00d4091540 msgr2=0x7f00d4091950 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:20.590 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.579+0000 7f00da007700 1 -- 172.21.15.170:0/1450447090 shutdown_connections 2020-03-19T21:18:20.590 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.579+0000 7f00da007700 1 -- 172.21.15.170:0/1450447090 wait complete. 2020-03-19T21:18:20.591 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.579+0000 7f00da007700 1 Processor -- start 2020-03-19T21:18:20.591 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.580+0000 7f00da007700 1 -- start start 2020-03-19T21:18:20.591 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.580+0000 7f00da007700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f00d4066760 0x7f00d411fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:20.591 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.580+0000 7f00da007700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00d4121190 con 0x7f00d4090890 2020-03-19T21:18:20.591 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.580+0000 7f00da007700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00d411ccc0 con 0x7f00d4066760 2020-03-19T21:18:20.591 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.580+0000 7f00d37fe700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f00d4066760 0x7f00d411fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:20.591 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.580+0000 7f00d37fe700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f00d4066760 0x7f00d411fa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53064/0 (socket says 172.21.15.170:53064) 2020-03-19T21:18:20.592 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.580+0000 7f00d37fe700 1 -- 172.21.15.170:0/1072630267 learned_addr learned my addr 172.21.15.170:0/1072630267 (peer_addr_for_me v2:172.21.15.170:0/0) 2020-03-19T21:18:20.592 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.580+0000 7f00d37fe700 1 -- 172.21.15.170:0/1072630267 >> v1:172.21.15.170:6789/0 conn(0x7f00d4090890 legacy=0x7f00d41233b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:20.592 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.580+0000 7f00d0ff9700 1 -- 172.21.15.170:0/1072630267 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3604777355 0 0) 0x7f00d4121190 con 0x7f00d4090890 2020-03-19T21:18:20.592 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.580+0000 7f00d37fe700 1 -- 172.21.15.170:0/1072630267 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00d4120e80 con 0x7f00d4066760 2020-03-19T21:18:20.592 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.580+0000 7f00d37fe700 1 --2- 172.21.15.170:0/1072630267 >> v2:172.21.15.170:3300/0 conn(0x7f00d4066760 0x7f00d411fa00 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f00d4120360 tx=0x7f00c4007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:20.592 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.580+0000 7f00d37fe700 1 --2- 172.21.15.170:0/1072630267 >> v2:172.21.15.170:3300/0 conn(0x7f00d4066760 0x7f00d411fa00 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f00d4120360 tx=0x7f00c4007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:20.592 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.581+0000 7f00d37fe700 1 --2- 172.21.15.170:0/1072630267 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f00d4066760 0x7f00d411fa00 secure :-1 s=READY pgs=173 cs=0 l=1 rx=0x7f00d4120360 tx=0x7f00c4007a40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:20.592 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.581+0000 7f00d37fe700 1 --2- 172.21.15.170:0/1072630267 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f00d4066760 0x7f00d411fa00 secure :-1 s=THROTTLE_DONE pgs=173 cs=0 l=1 rx=0x7f00d4120360 tx=0x7f00c4007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:20.593 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.581+0000 7f00d0ff9700 1 -- 172.21.15.170:0/1072630267 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f00c4007150 con 0x7f00d4066760 2020-03-19T21:18:20.593 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.581+0000 7f00d37fe700 1 --2- 172.21.15.170:0/1072630267 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f00d4066760 0x7f00d411fa00 secure :-1 s=THROTTLE_DONE pgs=173 cs=0 l=1 rx=0x7f00d4120360 tx=0x7f00c4007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:20.593 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.581+0000 7f00d37fe700 1 --2- 172.21.15.170:0/1072630267 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f00d4066760 0x7f00d411fa00 secure :-1 s=THROTTLE_DONE pgs=173 cs=0 l=1 rx=0x7f00d4120360 tx=0x7f00c4007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:20.593 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.581+0000 7f00d0ff9700 1 -- 172.21.15.170:0/1072630267 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00c40072b0 con 0x7f00d4066760 2020-03-19T21:18:20.593 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.581+0000 7f00da007700 1 -- 172.21.15.170:0/1072630267 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00d411e500 con 0x7f00d4066760 2020-03-19T21:18:20.593 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.581+0000 7f00d0ff9700 1 -- 172.21.15.170:0/1072630267 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f00c4007550 con 0x7f00d4066760 2020-03-19T21:18:20.593 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.581+0000 7f00da007700 1 -- 172.21.15.170:0/1072630267 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00d4123c80 con 0x7f00d4066760 2020-03-19T21:18:20.594 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.581+0000 7f00d37fe700 1 --2- 172.21.15.170:0/1072630267 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f00d4066760 0x7f00d411fa00 secure :-1 s=THROTTLE_DONE pgs=173 cs=0 l=1 rx=0x7f00d4120360 tx=0x7f00c4007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:20.594 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.581+0000 7f00d0ff9700 1 -- 172.21.15.170:0/1072630267 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 4) v1 ==== 29603+0+0 (secure 0 0 0) 0x7f00c40056c0 con 0x7f00d4066760 2020-03-19T21:18:20.594 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.581+0000 7f00d37fe700 1 --2- 172.21.15.170:0/1072630267 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f00d4066760 0x7f00d411fa00 secure :-1 s=THROTTLE_DONE pgs=173 cs=0 l=1 rx=0x7f00d4120360 tx=0x7f00c4007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:20.594 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.581+0000 7f00d0ff9700 1 --2- 172.21.15.170:0/1072630267 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f00b40282e0 0x7f00b402a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:20.594 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.581+0000 7f00d0ff9700 1 -- 172.21.15.170:0/1072630267 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3403+0+0 (secure 0 0 0) 0x7f00c40348f0 con 0x7f00d4066760 2020-03-19T21:18:20.594 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.582+0000 7f00d2ffd700 1 --2- 172.21.15.170:0/1072630267 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f00b40282e0 0x7f00b402a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:20.594 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.582+0000 7f00da007700 1 -- 172.21.15.170:0/1072630267 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00d411f270 con 0x7f00d4066760 2020-03-19T21:18:20.595 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.582+0000 7f00d2ffd700 1 --2- 172.21.15.170:0/1072630267 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f00b40282e0 0x7f00b402a7a0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f00c8001110 tx=0x7f00c8001670).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:20.595 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.582+0000 7f00d2ffd700 1 --2- 172.21.15.170:0/1072630267 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f00b40282e0 0x7f00b402a7a0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f00c8001110 tx=0x7f00c8001670).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:20.595 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.582+0000 7f00d2ffd700 1 --2- 172.21.15.170:0/1072630267 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f00b40282e0 0x7f00b402a7a0 secure :-1 s=READY pgs=43 cs=0 l=1 rx=0x7f00c8001110 tx=0x7f00c8001670).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:20.595 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.585+0000 7f00d37fe700 1 --2- 172.21.15.170:0/1072630267 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f00d4066760 0x7f00d411fa00 secure :-1 s=THROTTLE_DONE pgs=173 cs=0 l=1 rx=0x7f00d4120360 tx=0x7f00c4007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:20.595 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.585+0000 7f00d0ff9700 1 -- 172.21.15.170:0/1072630267 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+126772 (secure 0 0 0) 0x7f00c4006250 con 0x7f00d4066760 2020-03-19T21:18:20.721 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.718+0000 7f00da007700 1 -- 172.21.15.170:0/1072630267 --> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f00d4056db0 con 0x7f00b40282e0 2020-03-19T21:18:20.726 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.720+0000 7f00d2ffd700 1 --2- 172.21.15.170:0/1072630267 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f00b40282e0 0x7f00b402a7a0 secure :-1 s=THROTTLE_DONE pgs=43 cs=0 l=1 rx=0x7f00c8001110 tx=0x7f00c8001670).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:20.727 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.720+0000 7f00d0ff9700 1 -- 172.21.15.170:0/1072630267 <== mgr.4102 v2:172.21.15.170:6824/20154 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+63967 (secure 0 0 0) 0x7f00d4056db0 con 0x7f00b40282e0 2020-03-19T21:18:20.727 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.722+0000 7f00da007700 1 -- 172.21.15.170:0/1072630267 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f00b40282e0 msgr2=0x7f00b402a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:20.727 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.722+0000 7f00da007700 1 --2- 172.21.15.170:0/1072630267 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f00b40282e0 0x7f00b402a7a0 secure :-1 s=READY pgs=43 cs=0 l=1 rx=0x7f00c8001110 tx=0x7f00c8001670).stop 2020-03-19T21:18:20.727 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.722+0000 7f00da007700 1 -- 172.21.15.170:0/1072630267 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f00d4066760 msgr2=0x7f00d411fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:20.727 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.722+0000 7f00da007700 1 --2- 172.21.15.170:0/1072630267 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f00d4066760 0x7f00d411fa00 secure :-1 s=READY pgs=173 cs=0 l=1 rx=0x7f00d4120360 tx=0x7f00c4007a40).stop 2020-03-19T21:18:20.727 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.722+0000 7f00da007700 1 -- 172.21.15.170:0/1072630267 shutdown_connections 2020-03-19T21:18:20.728 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.722+0000 7f00da007700 1 --2- 172.21.15.170:0/1072630267 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f00b40282e0 0x7f00b402a7a0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:20.728 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.722+0000 7f00da007700 1 --2- 172.21.15.170:0/1072630267 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f00d4066760 0x7f00d411fa00 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:20.728 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.722+0000 7f00da007700 1 -- 172.21.15.170:0/1072630267 >> 172.21.15.170:0/1072630267 conn(0x7f00d4091540 msgr2=0x7f00d4093690 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:20.728 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.722+0000 7f00da007700 1 -- 172.21.15.170:0/1072630267 shutdown_connections 2020-03-19T21:18:20.728 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.722+0000 7f00da007700 1 -- 172.21.15.170:0/1072630267 wait complete. 2020-03-19T21:18:20.728 INFO:teuthology.orchestra.run.smithi170.stderr:dumped all 2020-03-19T21:18:20.729 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:18:20.734 INFO:teuthology.orchestra.run.smithi170.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2020-03-19T21:18:20.510144+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2286,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":314572800,"kb_used":3146224,"kb_used_data":472,"kb_used_omap":0,"kb_used_meta":3145728,"kb_avail":311426576,"statfs":{"total":322122547200,"available":318900813824,"internally_reserved":3221225472,"allocated":483328,"data_stored":132129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":3221225472},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.000344"},"pg_stats":[{"pgid":"3.7","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2020-03-19T21:18:13.525855+0000","last_change":"2020-03-19T21:18:13.525398+0000","last_active":"2020-03-19T21:18:13.525855+0000","last_peered":"2020-03-19T21:18:13.525855+0000","last_clean":"2020-03-19T21:18:13.525855+0000","last_became_active":"2020-03-19T21:18:13.525041+0000","last_became_peered":"2020-03-19T21:18:13.525041+0000","last_unstale":"2020-03-19T21:18:13.525855+0000","last_undegraded":"2020-03-19T21:18:13.525855+0000","last_fullsized":"2020-03-19T21:18:13.525855+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:13.185694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:13.185694+0000","last_clean_scrub_stamp":"2020-03-19T21:18:13.185694+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":"13","reported_epoch":"15","state":"active+clean","last_fresh":"2020-03-19T21:18:13.521281+0000","last_change":"2020-03-19T21:18:10.186965+0000","last_active":"2020-03-19T21:18:13.521281+0000","last_peered":"2020-03-19T21:18:13.521281+0000","last_clean":"2020-03-19T21:18:13.521281+0000","last_became_active":"2020-03-19T21:18:10.186584+0000","last_became_peered":"2020-03-19T21:18:10.186584+0000","last_unstale":"2020-03-19T21:18:13.521281+0000","last_undegraded":"2020-03-19T21:18:13.521281+0000","last_fullsized":"2020-03-19T21:18:13.521281+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:09.169195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:09.169195+0000","last_clean_scrub_stamp":"2020-03-19T21:18:09.169195+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"16'3","reported_seq":"13","reported_epoch":"16","state":"active+clean","last_fresh":"2020-03-19T21:18:16.214216+0000","last_change":"2020-03-19T21:18:13.197989+0000","last_active":"2020-03-19T21:18:16.214216+0000","last_peered":"2020-03-19T21:18:16.214216+0000","last_clean":"2020-03-19T21:18:16.214216+0000","last_became_active":"2020-03-19T21:18:13.197671+0000","last_became_peered":"2020-03-19T21:18:13.197671+0000","last_unstale":"2020-03-19T21:18:16.214216+0000","last_undegraded":"2020-03-19T21:18:16.214216+0000","last_fullsized":"2020-03-19T21:18:16.214216+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:12.182524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:12.182524+0000","last_clean_scrub_stamp":"2020-03-19T21:18:12.182524+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2020-03-19T21:18:13.526546+0000","last_change":"2020-03-19T21:18:13.526027+0000","last_active":"2020-03-19T21:18:13.526546+0000","last_peered":"2020-03-19T21:18:13.526546+0000","last_clean":"2020-03-19T21:18:13.526546+0000","last_became_active":"2020-03-19T21:18:13.524563+0000","last_became_peered":"2020-03-19T21:18:13.524563+0000","last_unstale":"2020-03-19T21:18:13.526546+0000","last_undegraded":"2020-03-19T21:18:13.526546+0000","last_fullsized":"2020-03-19T21:18:13.526546+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:13.185694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:13.185694+0000","last_clean_scrub_stamp":"2020-03-19T21:18:13.185694+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":"12","reported_epoch":"15","state":"active+clean","last_fresh":"2020-03-19T21:18:13.524035+0000","last_change":"2020-03-19T21:18:10.184204+0000","last_active":"2020-03-19T21:18:13.524035+0000","last_peered":"2020-03-19T21:18:13.524035+0000","last_clean":"2020-03-19T21:18:13.524035+0000","last_became_active":"2020-03-19T21:18:10.183810+0000","last_became_peered":"2020-03-19T21:18:10.183810+0000","last_unstale":"2020-03-19T21:18:13.524035+0000","last_undegraded":"2020-03-19T21:18:13.524035+0000","last_fullsized":"2020-03-19T21:18:13.524035+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:09.169195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:09.169195+0000","last_clean_scrub_stamp":"2020-03-19T21:18:09.169195+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"16'7","reported_seq":"17","reported_epoch":"16","state":"active+clean","last_fresh":"2020-03-19T21:18:16.216574+0000","last_change":"2020-03-19T21:18:13.196212+0000","last_active":"2020-03-19T21:18:16.216574+0000","last_peered":"2020-03-19T21:18:16.216574+0000","last_clean":"2020-03-19T21:18:16.216574+0000","last_became_active":"2020-03-19T21:18:13.195896+0000","last_became_peered":"2020-03-19T21:18:13.195896+0000","last_unstale":"2020-03-19T21:18:16.216574+0000","last_undegraded":"2020-03-19T21:18:16.216574+0000","last_fullsized":"2020-03-19T21:18:16.216574+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:12.182524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:12.182524+0000","last_clean_scrub_stamp":"2020-03-19T21:18:12.182524+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":566,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2020-03-19T21:18:13.524356+0000","last_change":"2020-03-19T21:18:13.524166+0000","last_active":"2020-03-19T21:18:13.524356+0000","last_peered":"2020-03-19T21:18:13.524356+0000","last_clean":"2020-03-19T21:18:13.524356+0000","last_became_active":"2020-03-19T21:18:13.523806+0000","last_became_peered":"2020-03-19T21:18:13.523806+0000","last_unstale":"2020-03-19T21:18:13.524356+0000","last_undegraded":"2020-03-19T21:18:13.524356+0000","last_fullsized":"2020-03-19T21:18:13.524356+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:13.185694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:13.185694+0000","last_clean_scrub_stamp":"2020-03-19T21:18:13.185694+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":"12","reported_epoch":"15","state":"active+clean","last_fresh":"2020-03-19T21:18:13.524452+0000","last_change":"2020-03-19T21:18:10.184210+0000","last_active":"2020-03-19T21:18:13.524452+0000","last_peered":"2020-03-19T21:18:13.524452+0000","last_clean":"2020-03-19T21:18:13.524452+0000","last_became_active":"2020-03-19T21:18:10.183788+0000","last_became_peered":"2020-03-19T21:18:10.183788+0000","last_unstale":"2020-03-19T21:18:13.524452+0000","last_undegraded":"2020-03-19T21:18:13.524452+0000","last_fullsized":"2020-03-19T21:18:13.524452+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:09.169195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:09.169195+0000","last_clean_scrub_stamp":"2020-03-19T21:18:09.169195+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"16'8","reported_seq":"18","reported_epoch":"16","state":"active+clean","last_fresh":"2020-03-19T21:18:16.216506+0000","last_change":"2020-03-19T21:18:13.197291+0000","last_active":"2020-03-19T21:18:16.216506+0000","last_peered":"2020-03-19T21:18:16.216506+0000","last_clean":"2020-03-19T21:18:16.216506+0000","last_became_active":"2020-03-19T21:18:13.196823+0000","last_became_peered":"2020-03-19T21:18:13.196823+0000","last_unstale":"2020-03-19T21:18:16.216506+0000","last_undegraded":"2020-03-19T21:18:16.216506+0000","last_fullsized":"2020-03-19T21:18:16.216506+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:12.182524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:12.182524+0000","last_clean_scrub_stamp":"2020-03-19T21:18:12.182524+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2020-03-19T21:18:13.528411+0000","last_change":"2020-03-19T21:18:13.528239+0000","last_active":"2020-03-19T21:18:13.528411+0000","last_peered":"2020-03-19T21:18:13.528411+0000","last_clean":"2020-03-19T21:18:13.528411+0000","last_became_active":"2020-03-19T21:18:13.527916+0000","last_became_peered":"2020-03-19T21:18:13.527916+0000","last_unstale":"2020-03-19T21:18:13.528411+0000","last_undegraded":"2020-03-19T21:18:13.528411+0000","last_fullsized":"2020-03-19T21:18:13.528411+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:13.185694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:13.185694+0000","last_clean_scrub_stamp":"2020-03-19T21:18:13.185694+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":"12","reported_epoch":"15","state":"active+clean","last_fresh":"2020-03-19T21:18:13.519207+0000","last_change":"2020-03-19T21:18:10.187875+0000","last_active":"2020-03-19T21:18:13.519207+0000","last_peered":"2020-03-19T21:18:13.519207+0000","last_clean":"2020-03-19T21:18:13.519207+0000","last_became_active":"2020-03-19T21:18:10.187570+0000","last_became_peered":"2020-03-19T21:18:10.187570+0000","last_unstale":"2020-03-19T21:18:13.519207+0000","last_undegraded":"2020-03-19T21:18:13.519207+0000","last_fullsized":"2020-03-19T21:18:13.519207+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:09.169195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:09.169195+0000","last_clean_scrub_stamp":"2020-03-19T21:18:09.169195+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2020-03-19T21:18:13.521933+0000","last_change":"2020-03-19T21:18:13.197535+0000","last_active":"2020-03-19T21:18:13.521933+0000","last_peered":"2020-03-19T21:18:13.521933+0000","last_clean":"2020-03-19T21:18:13.521933+0000","last_became_active":"2020-03-19T21:18:13.197224+0000","last_became_peered":"2020-03-19T21:18:13.197224+0000","last_unstale":"2020-03-19T21:18:13.521933+0000","last_undegraded":"2020-03-19T21:18:13.521933+0000","last_fullsized":"2020-03-19T21:18:13.521933+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:12.182524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:12.182524+0000","last_clean_scrub_stamp":"2020-03-19T21:18:12.182524+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2020-03-19T21:18:13.524858+0000","last_change":"2020-03-19T21:18:13.524218+0000","last_active":"2020-03-19T21:18:13.524858+0000","last_peered":"2020-03-19T21:18:13.524858+0000","last_clean":"2020-03-19T21:18:13.524858+0000","last_became_active":"2020-03-19T21:18:13.523902+0000","last_became_peered":"2020-03-19T21:18:13.523902+0000","last_unstale":"2020-03-19T21:18:13.524858+0000","last_undegraded":"2020-03-19T21:18:13.524858+0000","last_fullsized":"2020-03-19T21:18:13.524858+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:13.185694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:13.185694+0000","last_clean_scrub_stamp":"2020-03-19T21:18:13.185694+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":"12","reported_epoch":"15","state":"active+clean","last_fresh":"2020-03-19T21:18:13.523332+0000","last_change":"2020-03-19T21:18:10.184801+0000","last_active":"2020-03-19T21:18:13.523332+0000","last_peered":"2020-03-19T21:18:13.523332+0000","last_clean":"2020-03-19T21:18:13.523332+0000","last_became_active":"2020-03-19T21:18:10.184433+0000","last_became_peered":"2020-03-19T21:18:10.184433+0000","last_unstale":"2020-03-19T21:18:13.523332+0000","last_undegraded":"2020-03-19T21:18:13.523332+0000","last_fullsized":"2020-03-19T21:18:13.523332+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:09.169195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:09.169195+0000","last_clean_scrub_stamp":"2020-03-19T21:18:09.169195+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"16'5","reported_seq":"15","reported_epoch":"16","state":"active+clean","last_fresh":"2020-03-19T21:18:16.213106+0000","last_change":"2020-03-19T21:18:13.195202+0000","last_active":"2020-03-19T21:18:16.213106+0000","last_peered":"2020-03-19T21:18:16.213106+0000","last_clean":"2020-03-19T21:18:16.213106+0000","last_became_active":"2020-03-19T21:18:13.194849+0000","last_became_peered":"2020-03-19T21:18:13.194849+0000","last_unstale":"2020-03-19T21:18:16.213106+0000","last_undegraded":"2020-03-19T21:18:16.213106+0000","last_fullsized":"2020-03-19T21:18:16.213106+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:12.182524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:12.182524+0000","last_clean_scrub_stamp":"2020-03-19T21:18:12.182524+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2020-03-19T21:18:13.523537+0000","last_change":"2020-03-19T21:18:13.523284+0000","last_active":"2020-03-19T21:18:13.523537+0000","last_peered":"2020-03-19T21:18:13.523537+0000","last_clean":"2020-03-19T21:18:13.523537+0000","last_became_active":"2020-03-19T21:18:13.522937+0000","last_became_peered":"2020-03-19T21:18:13.522937+0000","last_unstale":"2020-03-19T21:18:13.523537+0000","last_undegraded":"2020-03-19T21:18:13.523537+0000","last_fullsized":"2020-03-19T21:18:13.523537+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:13.185694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:13.185694+0000","last_clean_scrub_stamp":"2020-03-19T21:18:13.185694+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":"13","reported_epoch":"15","state":"active+clean","last_fresh":"2020-03-19T21:18:13.521872+0000","last_change":"2020-03-19T21:18:10.183717+0000","last_active":"2020-03-19T21:18:13.521872+0000","last_peered":"2020-03-19T21:18:13.521872+0000","last_clean":"2020-03-19T21:18:13.521872+0000","last_became_active":"2020-03-19T21:18:10.183404+0000","last_became_peered":"2020-03-19T21:18:10.183404+0000","last_unstale":"2020-03-19T21:18:13.521872+0000","last_undegraded":"2020-03-19T21:18:13.521872+0000","last_fullsized":"2020-03-19T21:18:13.521872+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:09.169195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:09.169195+0000","last_clean_scrub_stamp":"2020-03-19T21:18:09.169195+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"16'1","reported_seq":"11","reported_epoch":"16","state":"active+clean","last_fresh":"2020-03-19T21:18:14.518247+0000","last_change":"2020-03-19T21:18:13.198955+0000","last_active":"2020-03-19T21:18:14.518247+0000","last_peered":"2020-03-19T21:18:14.518247+0000","last_clean":"2020-03-19T21:18:14.518247+0000","last_became_active":"2020-03-19T21:18:13.198498+0000","last_became_peered":"2020-03-19T21:18:13.198498+0000","last_unstale":"2020-03-19T21:18:14.518247+0000","last_undegraded":"2020-03-19T21:18:14.518247+0000","last_fullsized":"2020-03-19T21:18:14.518247+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:12.182524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:12.182524+0000","last_clean_scrub_stamp":"2020-03-19T21:18:12.182524+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2020-03-19T21:18:13.526493+0000","last_change":"2020-03-19T21:18:13.524754+0000","last_active":"2020-03-19T21:18:13.526493+0000","last_peered":"2020-03-19T21:18:13.526493+0000","last_clean":"2020-03-19T21:18:13.526493+0000","last_became_active":"2020-03-19T21:18:13.524437+0000","last_became_peered":"2020-03-19T21:18:13.524437+0000","last_unstale":"2020-03-19T21:18:13.526493+0000","last_undegraded":"2020-03-19T21:18:13.526493+0000","last_fullsized":"2020-03-19T21:18:13.526493+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:13.185694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:13.185694+0000","last_clean_scrub_stamp":"2020-03-19T21:18:13.185694+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":"12","reported_epoch":"15","state":"active+clean","last_fresh":"2020-03-19T21:18:13.524572+0000","last_change":"2020-03-19T21:18:10.184911+0000","last_active":"2020-03-19T21:18:13.524572+0000","last_peered":"2020-03-19T21:18:13.524572+0000","last_clean":"2020-03-19T21:18:13.524572+0000","last_became_active":"2020-03-19T21:18:10.184549+0000","last_became_peered":"2020-03-19T21:18:10.184549+0000","last_unstale":"2020-03-19T21:18:13.524572+0000","last_undegraded":"2020-03-19T21:18:13.524572+0000","last_fullsized":"2020-03-19T21:18:13.524572+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:09.169195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:09.169195+0000","last_clean_scrub_stamp":"2020-03-19T21:18:09.169195+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"16'6","reported_seq":"16","reported_epoch":"16","state":"active+clean","last_fresh":"2020-03-19T21:18:16.216619+0000","last_change":"2020-03-19T21:18:13.196469+0000","last_active":"2020-03-19T21:18:16.216619+0000","last_peered":"2020-03-19T21:18:16.216619+0000","last_clean":"2020-03-19T21:18:16.216619+0000","last_became_active":"2020-03-19T21:18:13.195947+0000","last_became_peered":"2020-03-19T21:18:13.195947+0000","last_unstale":"2020-03-19T21:18:16.216619+0000","last_undegraded":"2020-03-19T21:18:16.216619+0000","last_fullsized":"2020-03-19T21:18:16.216619+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:12.182524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:12.182524+0000","last_clean_scrub_stamp":"2020-03-19T21:18:12.182524+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":160,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2020-03-19T21:18:13.525722+0000","last_change":"2020-03-19T21:18:13.525236+0000","last_active":"2020-03-19T21:18:13.525722+0000","last_peered":"2020-03-19T21:18:13.525722+0000","last_clean":"2020-03-19T21:18:13.525722+0000","last_became_active":"2020-03-19T21:18:13.524938+0000","last_became_peered":"2020-03-19T21:18:13.524938+0000","last_unstale":"2020-03-19T21:18:13.525722+0000","last_undegraded":"2020-03-19T21:18:13.525722+0000","last_fullsized":"2020-03-19T21:18:13.525722+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:13.185694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:13.185694+0000","last_clean_scrub_stamp":"2020-03-19T21:18:13.185694+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":"12","reported_epoch":"15","state":"active+clean","last_fresh":"2020-03-19T21:18:13.523815+0000","last_change":"2020-03-19T21:18:10.182075+0000","last_active":"2020-03-19T21:18:13.523815+0000","last_peered":"2020-03-19T21:18:13.523815+0000","last_clean":"2020-03-19T21:18:13.523815+0000","last_became_active":"2020-03-19T21:18:10.181738+0000","last_became_peered":"2020-03-19T21:18:10.181738+0000","last_unstale":"2020-03-19T21:18:13.523815+0000","last_undegraded":"2020-03-19T21:18:13.523815+0000","last_fullsized":"2020-03-19T21:18:13.523815+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:09.169195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:09.169195+0000","last_clean_scrub_stamp":"2020-03-19T21:18:09.169195+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"16'8","reported_seq":"18","reported_epoch":"16","state":"active+clean","last_fresh":"2020-03-19T21:18:16.217013+0000","last_change":"2020-03-19T21:18:13.197376+0000","last_active":"2020-03-19T21:18:16.217013+0000","last_peered":"2020-03-19T21:18:16.217013+0000","last_clean":"2020-03-19T21:18:16.217013+0000","last_became_active":"2020-03-19T21:18:13.196089+0000","last_became_peered":"2020-03-19T21:18:13.196089+0000","last_unstale":"2020-03-19T21:18:16.217013+0000","last_undegraded":"2020-03-19T21:18:16.217013+0000","last_fullsized":"2020-03-19T21:18:16.217013+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:12.182524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:12.182524+0000","last_clean_scrub_stamp":"2020-03-19T21:18:12.182524+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1526,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":2286,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738374,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":1048724,"kb_used_data":140,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":103808876,"statfs":{"total":107374182400,"available":106300289024,"internally_reserved":1073741824,"allocated":143360,"data_stored":42553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47399999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61499999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738374,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":1048748,"kb_used_data":164,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":103808852,"statfs":{"total":107374182400,"available":106300264448,"internally_reserved":1073741824,"allocated":167936,"data_stored":44771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78400000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.8149999999999999}]}]},{"osd":1,"up_from":8,"seq":34359738373,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":1048752,"kb_used_data":168,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":103808848,"statfs":{"total":107374182400,"available":106300260352,"internally_reserved":1073741824,"allocated":172032,"data_stored":44805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.432}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58299999999999996}]}]}]}} 2020-03-19T21:18:20.737 INFO:tasks.ceph.ceph_manager.ceph:clean! 2020-03-19T21:18:20.737 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2020-03-19T21:18:20.737 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2020-03-19T21:18:20.738 INFO:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early health --format=json 2020-03-19T21:18:20.842 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.840+0000 7f35f5a8a700 1 Processor -- start 2020-03-19T21:18:20.842 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.840+0000 7f35f5a8a700 1 -- start start 2020-03-19T21:18:20.848 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.840+0000 7f35f5a8a700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f35f0090890 0x7f35f0128dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T21:18:20.848 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.840+0000 7f35f5a8a700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35f008ddb0 con 0x7f35f0066760 2020-03-19T21:18:20.848 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.840+0000 7f35f5a8a700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35f008d1b0 con 0x7f35f0090890 2020-03-19T21:18:20.848 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.840+0000 7f35eeffd700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f35f0090890 0x7f35f0128dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:20.848 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.840+0000 7f35ef7fe700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f35f0066760 0x7f35f0066b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:53834/0 (socket says 172.21.15.170:53834) 2020-03-19T21:18:20.849 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.840+0000 7f35ef7fe700 1 -- 172.21.15.170:0/3006729746 learned_addr learned my addr 172.21.15.170:0/3006729746 (peer_addr_for_me v1:172.21.15.170:0/0) 2020-03-19T21:18:20.849 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.840+0000 7f35eeffd700 1 -- 172.21.15.170:0/3006729746 >> v1:172.21.15.170:6789/0 conn(0x7f35f0066760 legacy=0x7f35f0066b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:20.849 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.840+0000 7f35eeffd700 1 -- 172.21.15.170:0/3006729746 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35f008d620 con 0x7f35f0090890 2020-03-19T21:18:20.849 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.840+0000 7f35ee7fc700 1 -- 172.21.15.170:0/3006729746 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2307744275 0 0) 0x7f35f008ddb0 con 0x7f35f0066760 2020-03-19T21:18:20.849 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.840+0000 7f35eeffd700 1 --2- 172.21.15.170:0/3006729746 >> v2:172.21.15.170:3300/0 conn(0x7f35f0090890 0x7f35f0128dd0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f35e4002f30 tx=0x7f35e4007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:20.849 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.840+0000 7f35eeffd700 1 --2- 172.21.15.170:0/3006729746 >> v2:172.21.15.170:3300/0 conn(0x7f35f0090890 0x7f35f0128dd0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f35e4002f30 tx=0x7f35e4007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:20.849 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.840+0000 7f35eeffd700 1 --2- 172.21.15.170:0/3006729746 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f35f0090890 0x7f35f0128dd0 secure :-1 s=READY pgs=175 cs=0 l=1 rx=0x7f35e4002f30 tx=0x7f35e4007a40).ready entity=mon.0 client_cookie=398e5ab3fb91125e server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:20.850 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.840+0000 7f35eeffd700 1 --2- 172.21.15.170:0/3006729746 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f35f0090890 0x7f35f0128dd0 secure :-1 s=THROTTLE_DONE pgs=175 cs=0 l=1 rx=0x7f35e4002f30 tx=0x7f35e4007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:20.850 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.840+0000 7f35ee7fc700 1 -- 172.21.15.170:0/3006729746 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f35e4008e80 con 0x7f35f0090890 2020-03-19T21:18:20.850 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.841+0000 7f35eeffd700 1 --2- 172.21.15.170:0/3006729746 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f35f0090890 0x7f35f0128dd0 secure :-1 s=THROTTLE_DONE pgs=175 cs=0 l=1 rx=0x7f35e4002f30 tx=0x7f35e4007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:20.852 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.841+0000 7f35eeffd700 1 --2- 172.21.15.170:0/3006729746 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f35f0090890 0x7f35f0128dd0 secure :-1 s=THROTTLE_DONE pgs=175 cs=0 l=1 rx=0x7f35e4002f30 tx=0x7f35e4007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:20.853 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.841+0000 7f35ee7fc700 1 -- 172.21.15.170:0/3006729746 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35e40113f0 con 0x7f35f0090890 2020-03-19T21:18:20.853 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.841+0000 7f35ee7fc700 1 -- 172.21.15.170:0/3006729746 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f35e40116d0 con 0x7f35f0090890 2020-03-19T21:18:20.853 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.841+0000 7f35f5a8a700 1 -- 172.21.15.170:0/3006729746 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f35f0090890 msgr2=0x7f35f0128dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:20.853 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.841+0000 7f35f5a8a700 1 --2- 172.21.15.170:0/3006729746 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f35f0090890 0x7f35f0128dd0 secure :-1 s=READY pgs=175 cs=0 l=1 rx=0x7f35e4002f30 tx=0x7f35e4007a40).stop 2020-03-19T21:18:20.853 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.841+0000 7f35f5a8a700 1 -- 172.21.15.170:0/3006729746 shutdown_connections 2020-03-19T21:18:20.853 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.841+0000 7f35f5a8a700 1 --2- 172.21.15.170:0/3006729746 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f35f0090890 0x7f35f0128dd0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:20.854 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.841+0000 7f35f5a8a700 1 -- 172.21.15.170:0/3006729746 >> 172.21.15.170:0/3006729746 conn(0x7f35f0091540 msgr2=0x7f35f0091950 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:20.854 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.841+0000 7f35f5a8a700 1 -- 172.21.15.170:0/3006729746 shutdown_connections 2020-03-19T21:18:20.854 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.841+0000 7f35f5a8a700 1 -- 172.21.15.170:0/3006729746 wait complete. 2020-03-19T21:18:20.854 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.843+0000 7f35f5a8a700 1 Processor -- start 2020-03-19T21:18:20.854 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.843+0000 7f35f5a8a700 1 -- start start 2020-03-19T21:18:20.854 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.843+0000 7f35f5a8a700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f35f0066760 0x7f35f0123400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:20.854 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.843+0000 7f35f5a8a700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35f01211e0 con 0x7f35f0090890 2020-03-19T21:18:20.855 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.843+0000 7f35f5a8a700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35f011f1e0 con 0x7f35f0066760 2020-03-19T21:18:20.855 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.843+0000 7f35ef7fe700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f35f0066760 0x7f35f0123400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:20.855 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.843+0000 7f35ef7fe700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f35f0066760 0x7f35f0123400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53074/0 (socket says 172.21.15.170:53074) 2020-03-19T21:18:20.855 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.843+0000 7f35eeffd700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f35f0090890 0x7f35f0123860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:53840/0 (socket says 172.21.15.170:53840) 2020-03-19T21:18:20.855 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.843+0000 7f35ef7fe700 1 -- 172.21.15.170:0/3033227353 learned_addr learned my addr 172.21.15.170:0/3033227353 (peer_addr_for_me v2:172.21.15.170:0/0) 2020-03-19T21:18:20.855 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.843+0000 7f35ef7fe700 1 -- 172.21.15.170:0/3033227353 >> v1:172.21.15.170:6789/0 conn(0x7f35f0090890 legacy=0x7f35f0123860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:20.856 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.843+0000 7f35ecff9700 1 -- 172.21.15.170:0/3033227353 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 930278636 0 0) 0x7f35f01211e0 con 0x7f35f0090890 2020-03-19T21:18:20.856 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.843+0000 7f35ef7fe700 1 -- 172.21.15.170:0/3033227353 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35f0120ed0 con 0x7f35f0066760 2020-03-19T21:18:20.856 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.843+0000 7f35ef7fe700 1 --2- 172.21.15.170:0/3033227353 >> v2:172.21.15.170:3300/0 conn(0x7f35f0066760 0x7f35f0123400 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f35f01202d0 tx=0x7f35e0010d10).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:20.856 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.843+0000 7f35ef7fe700 1 --2- 172.21.15.170:0/3033227353 >> v2:172.21.15.170:3300/0 conn(0x7f35f0066760 0x7f35f0123400 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f35f01202d0 tx=0x7f35e0010d10).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:20.856 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.843+0000 7f35ef7fe700 1 --2- 172.21.15.170:0/3033227353 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f35f0066760 0x7f35f0123400 secure :-1 s=READY pgs=177 cs=0 l=1 rx=0x7f35f01202d0 tx=0x7f35e0010d10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:20.856 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.843+0000 7f35ef7fe700 1 --2- 172.21.15.170:0/3033227353 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f35f0066760 0x7f35f0123400 secure :-1 s=THROTTLE_DONE pgs=177 cs=0 l=1 rx=0x7f35f01202d0 tx=0x7f35e0010d10).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:20.857 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.843+0000 7f35ecff9700 1 -- 172.21.15.170:0/3033227353 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f35e0012370 con 0x7f35f0066760 2020-03-19T21:18:20.857 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.843+0000 7f35ef7fe700 1 --2- 172.21.15.170:0/3033227353 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f35f0066760 0x7f35f0123400 secure :-1 s=THROTTLE_DONE pgs=177 cs=0 l=1 rx=0x7f35f01202d0 tx=0x7f35e0010d10).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:20.857 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.843+0000 7f35ecff9700 1 -- 172.21.15.170:0/3033227353 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35e00124d0 con 0x7f35f0066760 2020-03-19T21:18:20.857 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.843+0000 7f35ef7fe700 1 --2- 172.21.15.170:0/3033227353 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f35f0066760 0x7f35f0123400 secure :-1 s=THROTTLE_DONE pgs=177 cs=0 l=1 rx=0x7f35f01202d0 tx=0x7f35e0010d10).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:20.857 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.843+0000 7f35f5a8a700 1 -- 172.21.15.170:0/3033227353 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35f011e470 con 0x7f35f0066760 2020-03-19T21:18:20.858 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.843+0000 7f35ecff9700 1 -- 172.21.15.170:0/3033227353 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f35e00127b0 con 0x7f35f0066760 2020-03-19T21:18:20.858 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.843+0000 7f35f5a8a700 1 -- 172.21.15.170:0/3033227353 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35f011fa50 con 0x7f35f0066760 2020-03-19T21:18:20.858 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.843+0000 7f35ef7fe700 1 --2- 172.21.15.170:0/3033227353 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f35f0066760 0x7f35f0123400 secure :-1 s=THROTTLE_DONE pgs=177 cs=0 l=1 rx=0x7f35f01202d0 tx=0x7f35e0010d10).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:20.858 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.843+0000 7f35ecff9700 1 -- 172.21.15.170:0/3033227353 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 4) v1 ==== 29603+0+0 (secure 0 0 0) 0x7f35e00129b0 con 0x7f35f0066760 2020-03-19T21:18:20.858 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.844+0000 7f35ef7fe700 1 --2- 172.21.15.170:0/3033227353 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f35f0066760 0x7f35f0123400 secure :-1 s=THROTTLE_DONE pgs=177 cs=0 l=1 rx=0x7f35f01202d0 tx=0x7f35e0010d10).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:20.859 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.844+0000 7f35ecff9700 1 --2- 172.21.15.170:0/3033227353 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f35d0028240 0x7f35d002a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:20.859 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.844+0000 7f35ecff9700 1 -- 172.21.15.170:0/3033227353 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3403+0+0 (secure 0 0 0) 0x7f35e0034840 con 0x7f35f0066760 2020-03-19T21:18:20.859 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.844+0000 7f35eeffd700 1 --2- 172.21.15.170:0/3033227353 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f35d0028240 0x7f35d002a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:20.859 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.844+0000 7f35f5a8a700 1 -- 172.21.15.170:0/3033227353 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35f0056db0 con 0x7f35f0066760 2020-03-19T21:18:20.859 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.844+0000 7f35eeffd700 1 --2- 172.21.15.170:0/3033227353 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f35d0028240 0x7f35d002a700 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f35e4000b80 tx=0x7f35e4007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:20.860 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.844+0000 7f35eeffd700 1 --2- 172.21.15.170:0/3033227353 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f35d0028240 0x7f35d002a700 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f35e4000b80 tx=0x7f35e4007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:20.860 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.844+0000 7f35eeffd700 1 --2- 172.21.15.170:0/3033227353 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f35d0028240 0x7f35d002a700 secure :-1 s=READY pgs=44 cs=0 l=1 rx=0x7f35e4000b80 tx=0x7f35e4007a40).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:20.860 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.847+0000 7f35ef7fe700 1 --2- 172.21.15.170:0/3033227353 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f35f0066760 0x7f35f0123400 secure :-1 s=THROTTLE_DONE pgs=177 cs=0 l=1 rx=0x7f35f01202d0 tx=0x7f35e0010d10).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:20.860 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:20.847+0000 7f35ecff9700 1 -- 172.21.15.170:0/3033227353 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+126772 (secure 0 0 0) 0x7f35e0011af0 con 0x7f35f0066760 2020-03-19T21:18:21.038 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.035+0000 7f35f5a8a700 1 -- 172.21.15.170:0/3033227353 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f35f0120cb0 con 0x7f35f0066760 2020-03-19T21:18:21.038 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.035+0000 7f35ef7fe700 1 --2- 172.21.15.170:0/3033227353 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f35f0066760 0x7f35f0123400 secure :-1 s=THROTTLE_DONE pgs=177 cs=0 l=1 rx=0x7f35f01202d0 tx=0x7f35e0010d10).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:21.039 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.035+0000 7f35ecff9700 1 -- 172.21.15.170:0/3033227353 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+45 (secure 0 0 0) 0x7f35e0011af0 con 0x7f35f0066760 2020-03-19T21:18:21.039 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:18:21.040 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.036+0000 7f35f5a8a700 1 -- 172.21.15.170:0/3033227353 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f35d0028240 msgr2=0x7f35d002a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:21.040 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.036+0000 7f35f5a8a700 1 --2- 172.21.15.170:0/3033227353 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f35d0028240 0x7f35d002a700 secure :-1 s=READY pgs=44 cs=0 l=1 rx=0x7f35e4000b80 tx=0x7f35e4007a40).stop 2020-03-19T21:18:21.040 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.036+0000 7f35f5a8a700 1 -- 172.21.15.170:0/3033227353 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f35f0066760 msgr2=0x7f35f0123400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:21.040 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.036+0000 7f35f5a8a700 1 --2- 172.21.15.170:0/3033227353 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f35f0066760 0x7f35f0123400 secure :-1 s=READY pgs=177 cs=0 l=1 rx=0x7f35f01202d0 tx=0x7f35e0010d10).stop 2020-03-19T21:18:21.040 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.037+0000 7f35f5a8a700 1 -- 172.21.15.170:0/3033227353 shutdown_connections 2020-03-19T21:18:21.040 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.037+0000 7f35f5a8a700 1 --2- 172.21.15.170:0/3033227353 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f35d0028240 0x7f35d002a700 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:21.041 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.037+0000 7f35f5a8a700 1 --2- 172.21.15.170:0/3033227353 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f35f0066760 0x7f35f0123400 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:21.041 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.037+0000 7f35f5a8a700 1 -- 172.21.15.170:0/3033227353 >> 172.21.15.170:0/3033227353 conn(0x7f35f0091540 msgr2=0x7f35f00935b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:21.041 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.037+0000 7f35f5a8a700 1 -- 172.21.15.170:0/3033227353 shutdown_connections 2020-03-19T21:18:21.041 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.037+0000 7f35f5a8a700 1 -- 172.21.15.170:0/3033227353 wait complete. 2020-03-19T21:18:21.049 INFO:teuthology.orchestra.run.smithi170.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2020-03-19T21:18:21.049 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2020-03-19T21:18:21.049 INFO:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-03-19T21:18:21.154 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.153+0000 7fc07cb6f700 1 Processor -- start 2020-03-19T21:18:21.154 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.153+0000 7fc07cb6f700 1 -- start start 2020-03-19T21:18:21.160 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.154+0000 7fc07cb6f700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fc0780667d0 0x7fc078066bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T21:18:21.160 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.154+0000 7fc07cb6f700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc078062590 con 0x7fc078067010 2020-03-19T21:18:21.161 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.154+0000 7fc07cb6f700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0780907d0 con 0x7fc0780667d0 2020-03-19T21:18:21.161 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.154+0000 7fc07659c700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fc0780667d0 0x7fc078066bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:21.161 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.154+0000 7fc075d9b700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fc078067010 0x7fc07808bef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:53844/0 (socket says 172.21.15.170:53844) 2020-03-19T21:18:21.161 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.154+0000 7fc07659c700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fc0780667d0 0x7fc078066bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53080/0 (socket says 172.21.15.170:53080) 2020-03-19T21:18:21.161 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.154+0000 7fc075d9b700 1 -- 172.21.15.170:0/1354805993 learned_addr learned my addr 172.21.15.170:0/1354805993 (peer_addr_for_me v1:172.21.15.170:0/0) 2020-03-19T21:18:21.161 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.154+0000 7fc07659c700 1 -- 172.21.15.170:0/1354805993 >> v1:172.21.15.170:6789/0 conn(0x7fc078067010 legacy=0x7fc07808bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:21.162 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.154+0000 7fc07559a700 1 -- 172.21.15.170:0/1354805993 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 702069565 0 0) 0x7fc078062590 con 0x7fc078067010 2020-03-19T21:18:21.162 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.154+0000 7fc07659c700 1 -- 172.21.15.170:0/1354805993 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc07808d290 con 0x7fc0780667d0 2020-03-19T21:18:21.162 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.154+0000 7fc07659c700 1 --2- 172.21.15.170:0/1354805993 >> v2:172.21.15.170:3300/0 conn(0x7fc0780667d0 0x7fc078066bb0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7fc06c002f30 tx=0x7fc06c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:21.162 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.154+0000 7fc07659c700 1 --2- 172.21.15.170:0/1354805993 >> v2:172.21.15.170:3300/0 conn(0x7fc0780667d0 0x7fc078066bb0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7fc06c002f30 tx=0x7fc06c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:21.162 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.154+0000 7fc07659c700 1 --2- 172.21.15.170:0/1354805993 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc0780667d0 0x7fc078066bb0 secure :-1 s=READY pgs=179 cs=0 l=1 rx=0x7fc06c002f30 tx=0x7fc06c007a40).ready entity=mon.0 client_cookie=c99405c903c2e65c server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:21.162 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.154+0000 7fc07659c700 1 --2- 172.21.15.170:0/1354805993 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc0780667d0 0x7fc078066bb0 secure :-1 s=THROTTLE_DONE pgs=179 cs=0 l=1 rx=0x7fc06c002f30 tx=0x7fc06c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:21.163 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.154+0000 7fc07559a700 1 -- 172.21.15.170:0/1354805993 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fc06c008e80 con 0x7fc0780667d0 2020-03-19T21:18:21.163 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.154+0000 7fc07659c700 1 --2- 172.21.15.170:0/1354805993 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc0780667d0 0x7fc078066bb0 secure :-1 s=THROTTLE_DONE pgs=179 cs=0 l=1 rx=0x7fc06c002f30 tx=0x7fc06c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:21.163 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.154+0000 7fc07659c700 1 --2- 172.21.15.170:0/1354805993 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc0780667d0 0x7fc078066bb0 secure :-1 s=THROTTLE_DONE pgs=179 cs=0 l=1 rx=0x7fc06c002f30 tx=0x7fc06c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:21.163 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.154+0000 7fc07559a700 1 -- 172.21.15.170:0/1354805993 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc06c0113f0 con 0x7fc0780667d0 2020-03-19T21:18:21.163 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.154+0000 7fc07559a700 1 -- 172.21.15.170:0/1354805993 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fc06c0116d0 con 0x7fc0780667d0 2020-03-19T21:18:21.163 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.154+0000 7fc07cb6f700 1 -- 172.21.15.170:0/1354805993 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc0780667d0 msgr2=0x7fc078066bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:21.163 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.154+0000 7fc07cb6f700 1 --2- 172.21.15.170:0/1354805993 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc0780667d0 0x7fc078066bb0 secure :-1 s=READY pgs=179 cs=0 l=1 rx=0x7fc06c002f30 tx=0x7fc06c007a40).stop 2020-03-19T21:18:21.164 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.155+0000 7fc07cb6f700 1 -- 172.21.15.170:0/1354805993 shutdown_connections 2020-03-19T21:18:21.164 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.155+0000 7fc07cb6f700 1 --2- 172.21.15.170:0/1354805993 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc0780667d0 0x7fc078066bb0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:21.164 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.155+0000 7fc07cb6f700 1 -- 172.21.15.170:0/1354805993 >> 172.21.15.170:0/1354805993 conn(0x7fc078091540 msgr2=0x7fc078091950 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:21.164 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.155+0000 7fc07cb6f700 1 -- 172.21.15.170:0/1354805993 shutdown_connections 2020-03-19T21:18:21.164 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.155+0000 7fc07cb6f700 1 -- 172.21.15.170:0/1354805993 wait complete. 2020-03-19T21:18:21.165 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.156+0000 7fc07cb6f700 1 Processor -- start 2020-03-19T21:18:21.165 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.156+0000 7fc07cb6f700 1 -- start start 2020-03-19T21:18:21.165 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.156+0000 7fc07cb6f700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fc078067010 0x7fc078122770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:21.165 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.156+0000 7fc07cb6f700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0780607c0 con 0x7fc0780667d0 2020-03-19T21:18:21.165 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.156+0000 7fc07cb6f700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0780620b0 con 0x7fc078067010 2020-03-19T21:18:21.165 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.156+0000 7fc07659c700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fc0780667d0 0x7fc078060e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:53848/0 (socket says 172.21.15.170:53848) 2020-03-19T21:18:21.165 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.156+0000 7fc075d9b700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fc078067010 0x7fc078122770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:21.166 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.156+0000 7fc07659c700 1 -- 172.21.15.170:0/1257804728 learned_addr learned my addr 172.21.15.170:0/1257804728 (peer_addr_for_me v1:172.21.15.170:0/0) 2020-03-19T21:18:21.166 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.156+0000 7fc075d9b700 1 -- 172.21.15.170:0/1257804728 >> v1:172.21.15.170:6789/0 conn(0x7fc0780667d0 legacy=0x7fc078060e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:21.166 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.156+0000 7fc0677fe700 1 -- 172.21.15.170:0/1257804728 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1619845714 0 0) 0x7fc0780607c0 con 0x7fc0780667d0 2020-03-19T21:18:21.166 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.156+0000 7fc075d9b700 1 -- 172.21.15.170:0/1257804728 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc078123100 con 0x7fc078067010 2020-03-19T21:18:21.166 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.156+0000 7fc075d9b700 1 --2- 172.21.15.170:0/1257804728 >> v2:172.21.15.170:3300/0 conn(0x7fc078067010 0x7fc078122770 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7fc068000b80 tx=0x7fc068010950).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:21.167 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.156+0000 7fc075d9b700 1 --2- 172.21.15.170:0/1257804728 >> v2:172.21.15.170:3300/0 conn(0x7fc078067010 0x7fc078122770 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7fc068000b80 tx=0x7fc068010950).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:21.167 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.156+0000 7fc075d9b700 1 --2- 172.21.15.170:0/1257804728 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc078067010 0x7fc078122770 secure :-1 s=READY pgs=181 cs=0 l=1 rx=0x7fc068000b80 tx=0x7fc068010950).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:21.167 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.157+0000 7fc075d9b700 1 --2- 172.21.15.170:0/1257804728 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc078067010 0x7fc078122770 secure :-1 s=THROTTLE_DONE pgs=181 cs=0 l=1 rx=0x7fc068000b80 tx=0x7fc068010950).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:21.167 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.157+0000 7fc0677fe700 1 -- 172.21.15.170:0/1257804728 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fc068005b20 con 0x7fc078067010 2020-03-19T21:18:21.167 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.157+0000 7fc075d9b700 1 --2- 172.21.15.170:0/1257804728 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc078067010 0x7fc078122770 secure :-1 s=THROTTLE_DONE pgs=181 cs=0 l=1 rx=0x7fc068000b80 tx=0x7fc068010950).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:21.167 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.157+0000 7fc07cb6f700 1 -- 172.21.15.170:0/1257804728 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc078124420 con 0x7fc078067010 2020-03-19T21:18:21.167 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.157+0000 7fc0677fe700 1 -- 172.21.15.170:0/1257804728 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc068005c80 con 0x7fc078067010 2020-03-19T21:18:21.168 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.157+0000 7fc075d9b700 1 --2- 172.21.15.170:0/1257804728 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc078067010 0x7fc078122770 secure :-1 s=THROTTLE_DONE pgs=181 cs=0 l=1 rx=0x7fc068000b80 tx=0x7fc068010950).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:21.168 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.157+0000 7fc0677fe700 1 -- 172.21.15.170:0/1257804728 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fc0680060c0 con 0x7fc078067010 2020-03-19T21:18:21.168 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.157+0000 7fc07cb6f700 1 -- 172.21.15.170:0/1257804728 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0780617e0 con 0x7fc078067010 2020-03-19T21:18:21.168 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.157+0000 7fc075d9b700 1 --2- 172.21.15.170:0/1257804728 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc078067010 0x7fc078122770 secure :-1 s=THROTTLE_DONE pgs=181 cs=0 l=1 rx=0x7fc068000b80 tx=0x7fc068010950).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:21.168 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.157+0000 7fc0677fe700 1 -- 172.21.15.170:0/1257804728 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 4) v1 ==== 29603+0+0 (secure 0 0 0) 0x7fc06800a320 con 0x7fc078067010 2020-03-19T21:18:21.169 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.157+0000 7fc075d9b700 1 --2- 172.21.15.170:0/1257804728 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc078067010 0x7fc078122770 secure :-1 s=THROTTLE_DONE pgs=181 cs=0 l=1 rx=0x7fc068000b80 tx=0x7fc068010950).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:21.169 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.157+0000 7fc0677fe700 1 --2- 172.21.15.170:0/1257804728 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7fc054028230 0x7fc05402a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:21.169 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.157+0000 7fc0677fe700 1 -- 172.21.15.170:0/1257804728 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3403+0+0 (secure 0 0 0) 0x7fc068034000 con 0x7fc078067010 2020-03-19T21:18:21.169 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.157+0000 7fc07659c700 1 --2- 172.21.15.170:0/1257804728 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7fc054028230 0x7fc05402a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:21.169 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.158+0000 7fc07659c700 1 --2- 172.21.15.170:0/1257804728 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7fc054028230 0x7fc05402a6f0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7fc06c002ed0 tx=0x7fc06c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:21.169 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.158+0000 7fc07659c700 1 --2- 172.21.15.170:0/1257804728 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7fc054028230 0x7fc05402a6f0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7fc06c002ed0 tx=0x7fc06c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:21.170 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.158+0000 7fc07659c700 1 --2- 172.21.15.170:0/1257804728 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7fc054028230 0x7fc05402a6f0 secure :-1 s=READY pgs=45 cs=0 l=1 rx=0x7fc06c002ed0 tx=0x7fc06c007a40).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:21.170 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.158+0000 7fc07cb6f700 1 -- 172.21.15.170:0/1257804728 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc078056db0 con 0x7fc078067010 2020-03-19T21:18:21.177 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.161+0000 7fc075d9b700 1 --2- 172.21.15.170:0/1257804728 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc078067010 0x7fc078122770 secure :-1 s=THROTTLE_DONE pgs=181 cs=0 l=1 rx=0x7fc068000b80 tx=0x7fc068010950).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:21.177 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.161+0000 7fc0677fe700 1 -- 172.21.15.170:0/1257804728 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+126772 (secure 0 0 0) 0x7fc068007700 con 0x7fc078067010 2020-03-19T21:18:21.303 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.302+0000 7fc07cb6f700 1 -- 172.21.15.170:0/1257804728 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc0780678b0 con 0x7fc078067010 2020-03-19T21:18:21.304 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.303+0000 7fc075d9b700 1 --2- 172.21.15.170:0/1257804728 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc078067010 0x7fc078122770 secure :-1 s=THROTTLE_DONE pgs=181 cs=0 l=1 rx=0x7fc068000b80 tx=0x7fc068010950).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:21.304 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.303+0000 7fc0677fe700 1 -- 172.21.15.170:0/1257804728 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+8837 (secure 0 0 0) 0x7fc068011df0 con 0x7fc078067010 2020-03-19T21:18:21.305 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:18:21.305 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":16,"fsid":"12d0a33b-6b40-4398-90b4-6b5df1da388e","created":"2020-03-19T21:17:59.135747+0000","modified":"2020-03-19T21:18:14.515226+0000","last_up_change":"2020-03-19T21:18:05.158515+0000","last_in_change":"2020-03-19T21:18:05.158515+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":3,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2020-03-19T21:18:08.391604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2020-03-19T21:18:11.286536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2020-03-19T21:18:12.597110+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"32ca3517-1077-4102-9aa4-edfcebe8e3d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":20680},{"type":"v1","addr":"172.21.15.170:6801","nonce":20680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":20680},{"type":"v1","addr":"172.21.15.170:6803","nonce":20680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":20680},{"type":"v1","addr":"172.21.15.170:6807","nonce":20680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":20680},{"type":"v1","addr":"172.21.15.170:6805","nonce":20680}]},"public_addr":"172.21.15.170:6801/20680","cluster_addr":"172.21.15.170:6803/20680","heartbeat_back_addr":"172.21.15.170:6807/20680","heartbeat_front_addr":"172.21.15.170:6805/20680","state":["exists","up"]},{"osd":1,"uuid":"470df64b-60bc-4f5b-9b53-64495cd7c5e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":20681},{"type":"v1","addr":"172.21.15.170:6809","nonce":20681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":20681},{"type":"v1","addr":"172.21.15.170:6811","nonce":20681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":20681},{"type":"v1","addr":"172.21.15.170:6815","nonce":20681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":20681},{"type":"v1","addr":"172.21.15.170:6813","nonce":20681}]},"public_addr":"172.21.15.170:6809/20681","cluster_addr":"172.21.15.170:6811/20681","heartbeat_back_addr":"172.21.15.170:6815/20681","heartbeat_front_addr":"172.21.15.170:6813/20681","state":["exists","up"]},{"osd":2,"uuid":"ccfb9d82-b759-4ff4-8ddf-4f1d741bac5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":20682},{"type":"v1","addr":"172.21.15.170:6817","nonce":20682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":20682},{"type":"v1","addr":"172.21.15.170:6819","nonce":20682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":20682},{"type":"v1","addr":"172.21.15.170:6823","nonce":20682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":20682},{"type":"v1","addr":"172.21.15.170:6821","nonce":20682}]},"public_addr":"172.21.15.170:6817/20682","cluster_addr":"172.21.15.170:6819/20682","heartbeat_back_addr":"172.21.15.170:6823/20682","heartbeat_front_addr":"172.21.15.170:6821/20682","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2020-03-19T21:18:03.177293+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2020-03-19T21:18:03.263179+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2020-03-19T21:18:03.183223+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-03-19T21:18:21.306 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.304+0000 7fc07cb6f700 1 -- 172.21.15.170:0/1257804728 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7fc054028230 msgr2=0x7fc05402a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:21.306 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.304+0000 7fc07cb6f700 1 --2- 172.21.15.170:0/1257804728 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7fc054028230 0x7fc05402a6f0 secure :-1 s=READY pgs=45 cs=0 l=1 rx=0x7fc06c002ed0 tx=0x7fc06c007a40).stop 2020-03-19T21:18:21.306 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.304+0000 7fc07cb6f700 1 -- 172.21.15.170:0/1257804728 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc078067010 msgr2=0x7fc078122770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:21.306 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.304+0000 7fc07cb6f700 1 --2- 172.21.15.170:0/1257804728 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc078067010 0x7fc078122770 secure :-1 s=READY pgs=181 cs=0 l=1 rx=0x7fc068000b80 tx=0x7fc068010950).stop 2020-03-19T21:18:21.306 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.304+0000 7fc07cb6f700 1 -- 172.21.15.170:0/1257804728 shutdown_connections 2020-03-19T21:18:21.307 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.304+0000 7fc07cb6f700 1 --2- 172.21.15.170:0/1257804728 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7fc054028230 0x7fc05402a6f0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:21.307 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.304+0000 7fc07cb6f700 1 --2- 172.21.15.170:0/1257804728 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc078067010 0x7fc078122770 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:21.307 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.304+0000 7fc07cb6f700 1 -- 172.21.15.170:0/1257804728 >> 172.21.15.170:0/1257804728 conn(0x7fc078091540 msgr2=0x7fc078093680 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:21.307 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.304+0000 7fc07cb6f700 1 -- 172.21.15.170:0/1257804728 shutdown_connections 2020-03-19T21:18:21.307 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.304+0000 7fc07cb6f700 1 -- 172.21.15.170:0/1257804728 wait complete. 2020-03-19T21:18:21.316 INFO:tasks.cephfs.filesystem.ceph_manager:[{u'cache_target_full_ratio_micro': 800000, u'fast_read': False, u'stripe_width': 0, u'flags_names': u'hashpspool', u'tier_of': -1, u'hit_set_grade_decay_rate': 0, u'pg_placement_num': 8, u'use_gmt_hitset': True, u'cache_target_dirty_ratio_micro': 400000, u'create_time': u'2020-03-19T21:18:08.391604+0000', u'snap_epoch': 0, u'erasure_code_profile': u'', u'pg_autoscale_mode': u'off', u'snap_seq': 0, u'expected_num_objects': 0, u'size': 2, u'pg_num_pending': 8, u'auid': 0, u'cache_min_flush_age': 0, u'hit_set_period': 0, u'min_read_recency_for_promote': 0, u'target_max_objects': 0, u'pg_placement_num_target': 8, u'pg_num': 8, u'type': 1, u'grade_table': [], u'pool_name': u'rbd', u'cache_min_evict_age': 0, u'snap_mode': u'selfmanaged', u'pg_num_target': 8, u'cache_mode': u'none', u'min_size': 1, u'cache_target_dirty_high_ratio_micro': 600000, u'object_hash': 2, u'last_pg_merge_meta': {u'ready_epoch': 0, u'source_version': u"0'0", u'source_pgid': u'0.0', u'last_epoch_clean': 0, u'target_version': u"0'0", u'last_epoch_started': 0}, u'write_tier': -1, u'last_force_op_resend_preluminous': u'0', u'pool': 1, u'removed_snaps': u'[]', u'crush_rule': 0, u'tiers': [], u'hit_set_params': {u'type': u'none'}, u'last_force_op_resend': u'0', u'pool_snaps': [], u'quota_max_objects': 0, u'last_force_op_resend_prenautilus': u'0', u'application_metadata': {u'rbd': {}}, u'options': {}, u'hit_set_count': 0, u'flags': 1, u'target_max_bytes': 0, u'quota_max_bytes': 0, u'hit_set_search_last_n': 0, u'last_change': u'12', u'min_write_recency_for_promote': 0, u'read_tier': -1}, {u'cache_target_full_ratio_micro': 800000, u'fast_read': False, u'stripe_width': 0, u'flags_names': u'hashpspool', u'tier_of': -1, u'hit_set_grade_decay_rate': 0, u'pg_placement_num': 8, u'use_gmt_hitset': True, u'cache_target_dirty_ratio_micro': 400000, u'create_time': u'2020-03-19T21:18:11.286536+0000', u'snap_epoch': 0, u'erasure_code_profile': u'', u'pg_autoscale_mode': u'off', u'snap_seq': 0, u'expected_num_objects': 0, u'size': 2, u'pg_num_pending': 8, u'auid': 0, u'cache_min_flush_age': 0, u'hit_set_period': 0, u'min_read_recency_for_promote': 0, u'target_max_objects': 0, u'pg_placement_num_target': 8, u'pg_num': 8, u'type': 1, u'grade_table': [], u'pool_name': u'cephfs_metadata', u'cache_min_evict_age': 0, u'snap_mode': u'selfmanaged', u'pg_num_target': 8, u'cache_mode': u'none', u'min_size': 1, u'cache_target_dirty_high_ratio_micro': 600000, u'object_hash': 2, u'last_pg_merge_meta': {u'ready_epoch': 0, u'source_version': u"0'0", u'source_pgid': u'0.0', u'last_epoch_clean': 0, u'target_version': u"0'0", u'last_epoch_started': 0}, u'write_tier': -1, u'last_force_op_resend_preluminous': u'0', u'pool': 2, u'removed_snaps': u'[]', u'crush_rule': 0, u'tiers': [], u'hit_set_params': {u'type': u'none'}, u'last_force_op_resend': u'0', u'pool_snaps': [], u'quota_max_objects': 0, u'last_force_op_resend_prenautilus': u'0', u'application_metadata': {u'cephfs': {u'metadata': u'cephfs'}}, u'options': {u'pg_autoscale_bias': 4, u'recovery_priority': 5, u'pg_num_min': 16}, u'hit_set_count': 0, u'flags': 1, u'target_max_bytes': 0, u'quota_max_bytes': 0, u'hit_set_search_last_n': 0, u'last_change': u'15', u'min_write_recency_for_promote': 0, u'read_tier': -1}, {u'cache_target_full_ratio_micro': 800000, u'fast_read': False, u'stripe_width': 0, u'flags_names': u'hashpspool', u'tier_of': -1, u'hit_set_grade_decay_rate': 0, u'pg_placement_num': 8, u'use_gmt_hitset': True, u'cache_target_dirty_ratio_micro': 400000, u'create_time': u'2020-03-19T21:18:12.597110+0000', u'snap_epoch': 0, u'erasure_code_profile': u'', u'pg_autoscale_mode': u'off', u'snap_seq': 0, u'expected_num_objects': 0, u'size': 2, u'pg_num_pending': 8, u'auid': 0, u'cache_min_flush_age': 0, u'hit_set_period': 0, u'min_read_recency_for_promote': 0, u'target_max_objects': 0, u'pg_placement_num_target': 8, u'pg_num': 8, u'type': 1, u'grade_table': [], u'pool_name': u'cephfs_data', u'cache_min_evict_age': 0, u'snap_mode': u'selfmanaged', u'pg_num_target': 8, u'cache_mode': u'none', u'min_size': 1, u'cache_target_dirty_high_ratio_micro': 600000, u'object_hash': 2, u'last_pg_merge_meta': {u'ready_epoch': 0, u'source_version': u"0'0", u'source_pgid': u'0.0', u'last_epoch_clean': 0, u'target_version': u"0'0", u'last_epoch_started': 0}, u'write_tier': -1, u'last_force_op_resend_preluminous': u'0', u'pool': 3, u'removed_snaps': u'[]', u'crush_rule': 0, u'tiers': [], u'hit_set_params': {u'type': u'none'}, u'last_force_op_resend': u'0', u'pool_snaps': [], u'quota_max_objects': 0, u'last_force_op_resend_prenautilus': u'0', u'application_metadata': {u'cephfs': {u'data': u'cephfs'}}, u'options': {}, u'hit_set_count': 0, u'flags': 1, u'target_max_bytes': 0, u'quota_max_bytes': 0, u'hit_set_search_last_n': 0, u'last_change': u'16', u'min_write_recency_for_promote': 0, u'read_tier': -1}] 2020-03-19T21:18:21.316 INFO:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-03-19T21:18:21.424 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.423+0000 7f6f37fff700 1 Processor -- start 2020-03-19T21:18:21.425 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.423+0000 7f6f37fff700 1 -- start start 2020-03-19T21:18:21.430 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.424+0000 7f6f37fff700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f6f3808e5c0 0x7f6f38122960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T21:18:21.431 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.424+0000 7f6f37fff700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f38123270 con 0x7f6f38122dc0 2020-03-19T21:18:21.431 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.424+0000 7f6f37fff700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f381223b0 con 0x7f6f3808e5c0 2020-03-19T21:18:21.431 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.424+0000 7f6f36ffd700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f6f3808e5c0 0x7f6f38122960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:21.431 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.424+0000 7f6f36ffd700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f6f3808e5c0 0x7f6f38122960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53090/0 (socket says 172.21.15.170:53090) 2020-03-19T21:18:21.431 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.424+0000 7f6f367fc700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f6f38122dc0 0x7f6f38060070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:53856/0 (socket says 172.21.15.170:53856) 2020-03-19T21:18:21.432 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.424+0000 7f6f36ffd700 1 -- 172.21.15.170:0/510644666 learned_addr learned my addr 172.21.15.170:0/510644666 (peer_addr_for_me v2:172.21.15.170:0/0) 2020-03-19T21:18:21.432 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.424+0000 7f6f36ffd700 1 -- 172.21.15.170:0/510644666 >> v1:172.21.15.170:6789/0 conn(0x7f6f38122dc0 legacy=0x7f6f38060070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:21.432 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.424+0000 7f6f36ffd700 1 -- 172.21.15.170:0/510644666 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f38091f00 con 0x7f6f3808e5c0 2020-03-19T21:18:21.432 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.424+0000 7f6f36ffd700 1 --2- 172.21.15.170:0/510644666 >> v2:172.21.15.170:3300/0 conn(0x7f6f3808e5c0 0x7f6f38122960 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f6f2c002f30 tx=0x7f6f2c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:21.432 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.424+0000 7f6f36ffd700 1 --2- 172.21.15.170:0/510644666 >> v2:172.21.15.170:3300/0 conn(0x7f6f3808e5c0 0x7f6f38122960 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f6f2c002f30 tx=0x7f6f2c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:21.432 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.424+0000 7f6f36ffd700 1 --2- 172.21.15.170:0/510644666 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6f3808e5c0 0x7f6f38122960 secure :-1 s=READY pgs=183 cs=0 l=1 rx=0x7f6f2c002f30 tx=0x7f6f2c007a40).ready entity=mon.0 client_cookie=2da6a9af7b028d5b server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:21.433 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.424+0000 7f6f36ffd700 1 --2- 172.21.15.170:0/510644666 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6f3808e5c0 0x7f6f38122960 secure :-1 s=THROTTLE_DONE pgs=183 cs=0 l=1 rx=0x7f6f2c002f30 tx=0x7f6f2c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:21.433 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.424+0000 7f6f35ffb700 1 -- 172.21.15.170:0/510644666 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f6f2c0114b0 con 0x7f6f3808e5c0 2020-03-19T21:18:21.433 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.424+0000 7f6f36ffd700 1 --2- 172.21.15.170:0/510644666 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6f3808e5c0 0x7f6f38122960 secure :-1 s=THROTTLE_DONE pgs=183 cs=0 l=1 rx=0x7f6f2c002f30 tx=0x7f6f2c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:21.433 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.424+0000 7f6f35ffb700 1 -- 172.21.15.170:0/510644666 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f2c011610 con 0x7f6f3808e5c0 2020-03-19T21:18:21.433 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.424+0000 7f6f36ffd700 1 --2- 172.21.15.170:0/510644666 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6f3808e5c0 0x7f6f38122960 secure :-1 s=THROTTLE_DONE pgs=183 cs=0 l=1 rx=0x7f6f2c002f30 tx=0x7f6f2c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:21.434 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.424+0000 7f6f37fff700 1 -- 172.21.15.170:0/510644666 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6f3808e5c0 msgr2=0x7f6f38122960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:21.434 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.424+0000 7f6f35ffb700 1 -- 172.21.15.170:0/510644666 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f6f2c011970 con 0x7f6f3808e5c0 2020-03-19T21:18:21.434 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.424+0000 7f6f37fff700 1 --2- 172.21.15.170:0/510644666 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6f3808e5c0 0x7f6f38122960 secure :-1 s=READY pgs=183 cs=0 l=1 rx=0x7f6f2c002f30 tx=0x7f6f2c007a40).stop 2020-03-19T21:18:21.435 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.425+0000 7f6f37fff700 1 -- 172.21.15.170:0/510644666 shutdown_connections 2020-03-19T21:18:21.435 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.425+0000 7f6f37fff700 1 --2- 172.21.15.170:0/510644666 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6f3808e5c0 0x7f6f38122960 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:21.435 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.425+0000 7f6f37fff700 1 -- 172.21.15.170:0/510644666 >> 172.21.15.170:0/510644666 conn(0x7f6f380950e0 msgr2=0x7f6f38091180 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:21.435 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.425+0000 7f6f37fff700 1 -- 172.21.15.170:0/510644666 shutdown_connections 2020-03-19T21:18:21.435 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.425+0000 7f6f37fff700 1 -- 172.21.15.170:0/510644666 wait complete. 2020-03-19T21:18:21.435 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.426+0000 7f6f37fff700 1 Processor -- start 2020-03-19T21:18:21.436 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.426+0000 7f6f37fff700 1 -- start start 2020-03-19T21:18:21.436 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.426+0000 7f6f37fff700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f6f3808e5c0 0x7f6f38065ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:21.436 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.426+0000 7f6f37fff700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f3808fa20 con 0x7f6f38122dc0 2020-03-19T21:18:21.436 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.426+0000 7f6f37fff700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f3810eaf0 con 0x7f6f3808e5c0 2020-03-19T21:18:21.436 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.426+0000 7f6f36ffd700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f6f3808e5c0 0x7f6f38065ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:21.436 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.426+0000 7f6f36ffd700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f6f3808e5c0 0x7f6f38065ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53094/0 (socket says 172.21.15.170:53094) 2020-03-19T21:18:21.436 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.426+0000 7f6f36ffd700 1 -- 172.21.15.170:0/2342620932 learned_addr learned my addr 172.21.15.170:0/2342620932 (peer_addr_for_me v2:172.21.15.170:0/0) 2020-03-19T21:18:21.437 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.427+0000 7f6f36ffd700 1 -- 172.21.15.170:0/2342620932 >> v1:172.21.15.170:6789/0 conn(0x7f6f38122dc0 legacy=0x7f6f38065f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:21.437 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.427+0000 7f6f27fff700 1 -- 172.21.15.170:0/2342620932 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1366117893 0 0) 0x7f6f3808fa20 con 0x7f6f38122dc0 2020-03-19T21:18:21.437 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.427+0000 7f6f36ffd700 1 -- 172.21.15.170:0/2342620932 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f3810d4c0 con 0x7f6f3808e5c0 2020-03-19T21:18:21.437 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.427+0000 7f6f36ffd700 1 --2- 172.21.15.170:0/2342620932 >> v2:172.21.15.170:3300/0 conn(0x7f6f3808e5c0 0x7f6f38065ad0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f6f2c002f30 tx=0x7f6f2c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:21.438 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.427+0000 7f6f36ffd700 1 --2- 172.21.15.170:0/2342620932 >> v2:172.21.15.170:3300/0 conn(0x7f6f3808e5c0 0x7f6f38065ad0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f6f2c002f30 tx=0x7f6f2c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:21.449 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.427+0000 7f6f36ffd700 1 --2- 172.21.15.170:0/2342620932 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6f3808e5c0 0x7f6f38065ad0 secure :-1 s=READY pgs=185 cs=0 l=1 rx=0x7f6f2c002f30 tx=0x7f6f2c007a40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:21.449 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.427+0000 7f6f36ffd700 1 --2- 172.21.15.170:0/2342620932 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6f3808e5c0 0x7f6f38065ad0 secure :-1 s=THROTTLE_DONE pgs=185 cs=0 l=1 rx=0x7f6f2c002f30 tx=0x7f6f2c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:21.449 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.427+0000 7f6f27fff700 1 -- 172.21.15.170:0/2342620932 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f6f2c00f830 con 0x7f6f3808e5c0 2020-03-19T21:18:21.450 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.427+0000 7f6f36ffd700 1 --2- 172.21.15.170:0/2342620932 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6f3808e5c0 0x7f6f38065ad0 secure :-1 s=THROTTLE_DONE pgs=185 cs=0 l=1 rx=0x7f6f2c002f30 tx=0x7f6f2c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:21.450 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.427+0000 7f6f36ffd700 1 --2- 172.21.15.170:0/2342620932 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6f3808e5c0 0x7f6f38065ad0 secure :-1 s=THROTTLE_DONE pgs=185 cs=0 l=1 rx=0x7f6f2c002f30 tx=0x7f6f2c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:21.450 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.427+0000 7f6f27fff700 1 -- 172.21.15.170:0/2342620932 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f2c007150 con 0x7f6f3808e5c0 2020-03-19T21:18:21.451 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.427+0000 7f6f27fff700 1 -- 172.21.15.170:0/2342620932 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f6f2c0073f0 con 0x7f6f3808e5c0 2020-03-19T21:18:21.451 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.427+0000 7f6f37fff700 1 -- 172.21.15.170:0/2342620932 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f3811fa80 con 0x7f6f3808e5c0 2020-03-19T21:18:21.451 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.427+0000 7f6f37fff700 1 -- 172.21.15.170:0/2342620932 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f3810def0 con 0x7f6f3808e5c0 2020-03-19T21:18:21.451 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.427+0000 7f6f36ffd700 1 --2- 172.21.15.170:0/2342620932 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6f3808e5c0 0x7f6f38065ad0 secure :-1 s=THROTTLE_DONE pgs=185 cs=0 l=1 rx=0x7f6f2c002f30 tx=0x7f6f2c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:21.452 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.427+0000 7f6f27fff700 1 -- 172.21.15.170:0/2342620932 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 4) v1 ==== 29603+0+0 (secure 0 0 0) 0x7f6f2c0056c0 con 0x7f6f3808e5c0 2020-03-19T21:18:21.452 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.427+0000 7f6f36ffd700 1 --2- 172.21.15.170:0/2342620932 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6f3808e5c0 0x7f6f38065ad0 secure :-1 s=THROTTLE_DONE pgs=185 cs=0 l=1 rx=0x7f6f2c002f30 tx=0x7f6f2c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:21.453 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.428+0000 7f6f27fff700 1 --2- 172.21.15.170:0/2342620932 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f6f14028200 0x7f6f1402a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:21.454 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.428+0000 7f6f27fff700 1 -- 172.21.15.170:0/2342620932 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3403+0+0 (secure 0 0 0) 0x7f6f2c0348a0 con 0x7f6f3808e5c0 2020-03-19T21:18:21.454 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.428+0000 7f6f367fc700 1 --2- 172.21.15.170:0/2342620932 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f6f14028200 0x7f6f1402a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:21.454 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.428+0000 7f6f367fc700 1 --2- 172.21.15.170:0/2342620932 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f6f14028200 0x7f6f1402a6c0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f6f28001000 tx=0x7f6f280016b0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:21.455 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.428+0000 7f6f37fff700 1 -- 172.21.15.170:0/2342620932 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f38056db0 con 0x7f6f3808e5c0 2020-03-19T21:18:21.455 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.428+0000 7f6f367fc700 1 --2- 172.21.15.170:0/2342620932 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f6f14028200 0x7f6f1402a6c0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f6f28001000 tx=0x7f6f280016b0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:21.455 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.428+0000 7f6f367fc700 1 --2- 172.21.15.170:0/2342620932 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f6f14028200 0x7f6f1402a6c0 secure :-1 s=READY pgs=46 cs=0 l=1 rx=0x7f6f28001000 tx=0x7f6f280016b0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:21.457 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.431+0000 7f6f36ffd700 1 --2- 172.21.15.170:0/2342620932 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6f3808e5c0 0x7f6f38065ad0 secure :-1 s=THROTTLE_DONE pgs=185 cs=0 l=1 rx=0x7f6f2c002f30 tx=0x7f6f2c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:21.457 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.431+0000 7f6f27fff700 1 -- 172.21.15.170:0/2342620932 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+126772 (secure 0 0 0) 0x7f6f2c006610 con 0x7f6f3808e5c0 2020-03-19T21:18:21.572 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.571+0000 7f6f37fff700 1 -- 172.21.15.170:0/2342620932 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f6f3805e820 con 0x7f6f3808e5c0 2020-03-19T21:18:21.573 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.571+0000 7f6f36ffd700 1 --2- 172.21.15.170:0/2342620932 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6f3808e5c0 0x7f6f38065ad0 secure :-1 s=THROTTLE_DONE pgs=185 cs=0 l=1 rx=0x7f6f2c002f30 tx=0x7f6f2c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:21.573 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.571+0000 7f6f27fff700 1 -- 172.21.15.170:0/2342620932 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v16) v1 ==== 92+0+10 (secure 0 0 0) 0x7f6f2c006070 con 0x7f6f3808e5c0 2020-03-19T21:18:21.573 INFO:teuthology.orchestra.run.smithi170.stdout:pg_num: 8 2020-03-19T21:18:21.574 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.572+0000 7f6f37fff700 1 -- 172.21.15.170:0/2342620932 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f6f14028200 msgr2=0x7f6f1402a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:21.575 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.572+0000 7f6f37fff700 1 --2- 172.21.15.170:0/2342620932 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f6f14028200 0x7f6f1402a6c0 secure :-1 s=READY pgs=46 cs=0 l=1 rx=0x7f6f28001000 tx=0x7f6f280016b0).stop 2020-03-19T21:18:21.575 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.572+0000 7f6f37fff700 1 -- 172.21.15.170:0/2342620932 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6f3808e5c0 msgr2=0x7f6f38065ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:21.575 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.572+0000 7f6f37fff700 1 --2- 172.21.15.170:0/2342620932 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6f3808e5c0 0x7f6f38065ad0 secure :-1 s=READY pgs=185 cs=0 l=1 rx=0x7f6f2c002f30 tx=0x7f6f2c007a40).stop 2020-03-19T21:18:21.575 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.572+0000 7f6f37fff700 1 -- 172.21.15.170:0/2342620932 shutdown_connections 2020-03-19T21:18:21.575 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.572+0000 7f6f37fff700 1 --2- 172.21.15.170:0/2342620932 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f6f14028200 0x7f6f1402a6c0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:21.575 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.572+0000 7f6f37fff700 1 --2- 172.21.15.170:0/2342620932 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6f3808e5c0 0x7f6f38065ad0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:21.576 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.572+0000 7f6f37fff700 1 -- 172.21.15.170:0/2342620932 >> 172.21.15.170:0/2342620932 conn(0x7f6f380950e0 msgr2=0x7f6f38123720 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:21.576 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.572+0000 7f6f37fff700 1 -- 172.21.15.170:0/2342620932 shutdown_connections 2020-03-19T21:18:21.576 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.573+0000 7f6f37fff700 1 -- 172.21.15.170:0/2342620932 wait complete. 2020-03-19T21:18:21.584 INFO:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2020-03-19T21:18:21.691 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.690+0000 7f47c8349700 1 Processor -- start 2020-03-19T21:18:21.691 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.690+0000 7f47c8349700 1 -- start start 2020-03-19T21:18:21.697 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.690+0000 7f47c8349700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f47c0065970 0x7f47c0065d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T21:18:21.697 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.690+0000 7f47c8349700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47c0060f10 con 0x7f47c00661b0 2020-03-19T21:18:21.697 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.690+0000 7f47c8349700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47c0061490 con 0x7f47c0065970 2020-03-19T21:18:21.697 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.691+0000 7f47c60e5700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f47c0065970 0x7f47c0065d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:21.698 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.691+0000 7f47c60e5700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f47c0065970 0x7f47c0065d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53100/0 (socket says 172.21.15.170:53100) 2020-03-19T21:18:21.698 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.691+0000 7f47c58e4700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f47c00661b0 0x7f47c008f290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:53864/0 (socket says 172.21.15.170:53864) 2020-03-19T21:18:21.698 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.691+0000 7f47c60e5700 1 -- 172.21.15.170:0/373515802 learned_addr learned my addr 172.21.15.170:0/373515802 (peer_addr_for_me v2:172.21.15.170:0/0) 2020-03-19T21:18:21.698 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.691+0000 7f47c60e5700 1 -- 172.21.15.170:0/373515802 >> v1:172.21.15.170:6789/0 conn(0x7f47c00661b0 legacy=0x7f47c008f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:21.698 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.691+0000 7f47c60e5700 1 -- 172.21.15.170:0/373515802 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47c0061730 con 0x7f47c0065970 2020-03-19T21:18:21.698 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.691+0000 7f47c60e5700 1 --2- 172.21.15.170:0/373515802 >> v2:172.21.15.170:3300/0 conn(0x7f47c0065970 0x7f47c0065d50 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f47b80032b0 tx=0x7f47b8007dc0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:21.699 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.691+0000 7f47c60e5700 1 --2- 172.21.15.170:0/373515802 >> v2:172.21.15.170:3300/0 conn(0x7f47c0065970 0x7f47c0065d50 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f47b80032b0 tx=0x7f47b8007dc0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:21.699 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.691+0000 7f47c60e5700 1 --2- 172.21.15.170:0/373515802 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f47c0065970 0x7f47c0065d50 secure :-1 s=READY pgs=187 cs=0 l=1 rx=0x7f47b80032b0 tx=0x7f47b8007dc0).ready entity=mon.0 client_cookie=6b09a49d6007e411 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:21.699 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.691+0000 7f47c60e5700 1 --2- 172.21.15.170:0/373515802 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f47c0065970 0x7f47c0065d50 secure :-1 s=THROTTLE_DONE pgs=187 cs=0 l=1 rx=0x7f47b80032b0 tx=0x7f47b8007dc0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:21.699 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.691+0000 7f47c50e3700 1 -- 172.21.15.170:0/373515802 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f47b8011d90 con 0x7f47c0065970 2020-03-19T21:18:21.699 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.691+0000 7f47c60e5700 1 --2- 172.21.15.170:0/373515802 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f47c0065970 0x7f47c0065d50 secure :-1 s=THROTTLE_DONE pgs=187 cs=0 l=1 rx=0x7f47b80032b0 tx=0x7f47b8007dc0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:21.699 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.691+0000 7f47c60e5700 1 --2- 172.21.15.170:0/373515802 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f47c0065970 0x7f47c0065d50 secure :-1 s=THROTTLE_DONE pgs=187 cs=0 l=1 rx=0x7f47b80032b0 tx=0x7f47b8007dc0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:21.699 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.691+0000 7f47c50e3700 1 -- 172.21.15.170:0/373515802 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47b800a160 con 0x7f47c0065970 2020-03-19T21:18:21.700 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.691+0000 7f47c50e3700 1 -- 172.21.15.170:0/373515802 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f47b800a4c0 con 0x7f47c0065970 2020-03-19T21:18:21.700 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.691+0000 7f47c8349700 1 -- 172.21.15.170:0/373515802 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f47c0065970 msgr2=0x7f47c0065d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:21.700 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.691+0000 7f47c8349700 1 --2- 172.21.15.170:0/373515802 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f47c0065970 0x7f47c0065d50 secure :-1 s=READY pgs=187 cs=0 l=1 rx=0x7f47b80032b0 tx=0x7f47b8007dc0).stop 2020-03-19T21:18:21.700 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.692+0000 7f47c8349700 1 -- 172.21.15.170:0/373515802 shutdown_connections 2020-03-19T21:18:21.700 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.692+0000 7f47c8349700 1 --2- 172.21.15.170:0/373515802 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f47c0065970 0x7f47c0065d50 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:21.700 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.692+0000 7f47c8349700 1 -- 172.21.15.170:0/373515802 >> 172.21.15.170:0/373515802 conn(0x7f47c0091390 msgr2=0x7f47c0091770 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:21.700 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.692+0000 7f47c8349700 1 -- 172.21.15.170:0/373515802 shutdown_connections 2020-03-19T21:18:21.701 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.692+0000 7f47c8349700 1 -- 172.21.15.170:0/373515802 wait complete. 2020-03-19T21:18:21.701 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.692+0000 7f47c8349700 1 Processor -- start 2020-03-19T21:18:21.701 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.692+0000 7f47c8349700 1 -- start start 2020-03-19T21:18:21.701 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.693+0000 7f47c8349700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f47c0065970 0x7f47c0115760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:21.701 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.693+0000 7f47c8349700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47c01328b0 con 0x7f47c00661b0 2020-03-19T21:18:21.701 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.693+0000 7f47c8349700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47c01138c0 con 0x7f47c0065970 2020-03-19T21:18:21.701 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.693+0000 7f47c60e5700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f47c0065970 0x7f47c0115760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:21.702 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.693+0000 7f47c60e5700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f47c0065970 0x7f47c0115760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53104/0 (socket says 172.21.15.170:53104) 2020-03-19T21:18:21.702 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.693+0000 7f47c60e5700 1 -- 172.21.15.170:0/3014928740 learned_addr learned my addr 172.21.15.170:0/3014928740 (peer_addr_for_me v2:172.21.15.170:0/0) 2020-03-19T21:18:21.702 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.693+0000 7f47c58e4700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f47c00661b0 0x7f47c0132ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:53870/0 (socket says 172.21.15.170:53870) 2020-03-19T21:18:21.702 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.693+0000 7f47c60e5700 1 -- 172.21.15.170:0/3014928740 >> v1:172.21.15.170:6789/0 conn(0x7f47c00661b0 legacy=0x7f47c0132ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:21.702 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.693+0000 7f47c60e5700 1 -- 172.21.15.170:0/3014928740 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47c0115f00 con 0x7f47c0065970 2020-03-19T21:18:21.702 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.693+0000 7f47b37fe700 1 -- 172.21.15.170:0/3014928740 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3132897901 0 0) 0x7f47c01328b0 con 0x7f47c00661b0 2020-03-19T21:18:21.703 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.693+0000 7f47c60e5700 1 --2- 172.21.15.170:0/3014928740 >> v2:172.21.15.170:3300/0 conn(0x7f47c0065970 0x7f47c0115760 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f47b80032b0 tx=0x7f47b8007dc0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:21.703 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.693+0000 7f47c60e5700 1 --2- 172.21.15.170:0/3014928740 >> v2:172.21.15.170:3300/0 conn(0x7f47c0065970 0x7f47c0115760 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f47b80032b0 tx=0x7f47b8007dc0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:21.703 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.693+0000 7f47c60e5700 1 --2- 172.21.15.170:0/3014928740 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f47c0065970 0x7f47c0115760 secure :-1 s=READY pgs=189 cs=0 l=1 rx=0x7f47b80032b0 tx=0x7f47b8007dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:21.703 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.693+0000 7f47c60e5700 1 --2- 172.21.15.170:0/3014928740 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f47c0065970 0x7f47c0115760 secure :-1 s=THROTTLE_DONE pgs=189 cs=0 l=1 rx=0x7f47b80032b0 tx=0x7f47b8007dc0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:21.703 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.693+0000 7f47b37fe700 1 -- 172.21.15.170:0/3014928740 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f47b8005b70 con 0x7f47c0065970 2020-03-19T21:18:21.703 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.693+0000 7f47c60e5700 1 --2- 172.21.15.170:0/3014928740 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f47c0065970 0x7f47c0115760 secure :-1 s=THROTTLE_DONE pgs=189 cs=0 l=1 rx=0x7f47b80032b0 tx=0x7f47b8007dc0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:21.703 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.693+0000 7f47c60e5700 1 --2- 172.21.15.170:0/3014928740 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f47c0065970 0x7f47c0115760 secure :-1 s=THROTTLE_DONE pgs=189 cs=0 l=1 rx=0x7f47b80032b0 tx=0x7f47b8007dc0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:21.704 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.693+0000 7f47b37fe700 1 -- 172.21.15.170:0/3014928740 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47b8005cd0 con 0x7f47c0065970 2020-03-19T21:18:21.704 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.693+0000 7f47b37fe700 1 -- 172.21.15.170:0/3014928740 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f47b8007150 con 0x7f47c0065970 2020-03-19T21:18:21.704 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.693+0000 7f47c8349700 1 -- 172.21.15.170:0/3014928740 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47c0111e80 con 0x7f47c0065970 2020-03-19T21:18:21.704 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.693+0000 7f47c8349700 1 -- 172.21.15.170:0/3014928740 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47c01333b0 con 0x7f47c0065970 2020-03-19T21:18:21.704 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.694+0000 7f47c60e5700 1 --2- 172.21.15.170:0/3014928740 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f47c0065970 0x7f47c0115760 secure :-1 s=THROTTLE_DONE pgs=189 cs=0 l=1 rx=0x7f47b80032b0 tx=0x7f47b8007dc0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:21.704 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.694+0000 7f47b37fe700 1 -- 172.21.15.170:0/3014928740 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 4) v1 ==== 29603+0+0 (secure 0 0 0) 0x7f47b8007380 con 0x7f47c0065970 2020-03-19T21:18:21.704 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.694+0000 7f47c60e5700 1 --2- 172.21.15.170:0/3014928740 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f47c0065970 0x7f47c0115760 secure :-1 s=THROTTLE_DONE pgs=189 cs=0 l=1 rx=0x7f47b80032b0 tx=0x7f47b8007dc0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:21.705 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.694+0000 7f47b37fe700 1 --2- 172.21.15.170:0/3014928740 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f47a0028250 0x7f47a002a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:21.705 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.694+0000 7f47b37fe700 1 -- 172.21.15.170:0/3014928740 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3403+0+0 (secure 0 0 0) 0x7f47b80351a0 con 0x7f47c0065970 2020-03-19T21:18:21.705 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.694+0000 7f47c58e4700 1 --2- 172.21.15.170:0/3014928740 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f47a0028250 0x7f47a002a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:21.705 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.695+0000 7f47c58e4700 1 --2- 172.21.15.170:0/3014928740 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f47a0028250 0x7f47a002a710 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f47b4001000 tx=0x7f47b40016c0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:21.705 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.695+0000 7f47c8349700 1 -- 172.21.15.170:0/3014928740 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47c0114f00 con 0x7f47c0065970 2020-03-19T21:18:21.705 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.695+0000 7f47c58e4700 1 --2- 172.21.15.170:0/3014928740 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f47a0028250 0x7f47a002a710 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f47b4001000 tx=0x7f47b40016c0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:21.705 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.695+0000 7f47c58e4700 1 --2- 172.21.15.170:0/3014928740 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f47a0028250 0x7f47a002a710 secure :-1 s=READY pgs=47 cs=0 l=1 rx=0x7f47b4001000 tx=0x7f47b40016c0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:21.706 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.698+0000 7f47c60e5700 1 --2- 172.21.15.170:0/3014928740 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f47c0065970 0x7f47c0115760 secure :-1 s=THROTTLE_DONE pgs=189 cs=0 l=1 rx=0x7f47b80032b0 tx=0x7f47b8007dc0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:21.706 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.698+0000 7f47b37fe700 1 -- 172.21.15.170:0/3014928740 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+126772 (secure 0 0 0) 0x7f47b8006d70 con 0x7f47c0065970 2020-03-19T21:18:21.838 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.837+0000 7f47c8349700 1 -- 172.21.15.170:0/3014928740 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f47c0056db0 con 0x7f47c0065970 2020-03-19T21:18:21.839 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.837+0000 7f47c60e5700 1 --2- 172.21.15.170:0/3014928740 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f47c0065970 0x7f47c0115760 secure :-1 s=THROTTLE_DONE pgs=189 cs=0 l=1 rx=0x7f47b80032b0 tx=0x7f47b8007dc0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:21.839 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.837+0000 7f47b37fe700 1 -- 172.21.15.170:0/3014928740 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v16) v1 ==== 104+0+10 (secure 0 0 0) 0x7f47b8006d70 con 0x7f47c0065970 2020-03-19T21:18:21.839 INFO:teuthology.orchestra.run.smithi170.stdout:pg_num: 8 2020-03-19T21:18:21.840 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.838+0000 7f47c8349700 1 -- 172.21.15.170:0/3014928740 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f47a0028250 msgr2=0x7f47a002a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:21.840 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.839+0000 7f47c8349700 1 --2- 172.21.15.170:0/3014928740 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f47a0028250 0x7f47a002a710 secure :-1 s=READY pgs=47 cs=0 l=1 rx=0x7f47b4001000 tx=0x7f47b40016c0).stop 2020-03-19T21:18:21.841 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.839+0000 7f47c8349700 1 -- 172.21.15.170:0/3014928740 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f47c0065970 msgr2=0x7f47c0115760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:21.841 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.839+0000 7f47c8349700 1 --2- 172.21.15.170:0/3014928740 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f47c0065970 0x7f47c0115760 secure :-1 s=READY pgs=189 cs=0 l=1 rx=0x7f47b80032b0 tx=0x7f47b8007dc0).stop 2020-03-19T21:18:21.841 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.839+0000 7f47c8349700 1 -- 172.21.15.170:0/3014928740 shutdown_connections 2020-03-19T21:18:21.841 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.839+0000 7f47c8349700 1 --2- 172.21.15.170:0/3014928740 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f47a0028250 0x7f47a002a710 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:21.841 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.839+0000 7f47c8349700 1 --2- 172.21.15.170:0/3014928740 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f47c0065970 0x7f47c0115760 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:21.841 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.839+0000 7f47c8349700 1 -- 172.21.15.170:0/3014928740 >> 172.21.15.170:0/3014928740 conn(0x7f47c0091390 msgr2=0x7f47c0092fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:21.841 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.839+0000 7f47c8349700 1 -- 172.21.15.170:0/3014928740 shutdown_connections 2020-03-19T21:18:21.841 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.839+0000 7f47c8349700 1 -- 172.21.15.170:0/3014928740 wait complete. 2020-03-19T21:18:21.850 INFO:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2020-03-19T21:18:21.959 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.957+0000 7f89d9df9700 1 Processor -- start 2020-03-19T21:18:21.959 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.958+0000 7f89d9df9700 1 -- start start 2020-03-19T21:18:21.965 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.958+0000 7f89d9df9700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f89d4125840 0x7f89d4127c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T21:18:21.965 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.958+0000 7f89d9df9700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89d41228a0 con 0x7f89d41224c0 2020-03-19T21:18:21.965 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.958+0000 7f89d9df9700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89d4092200 con 0x7f89d4125840 2020-03-19T21:18:21.965 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.958+0000 7f89d2ffd700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f89d4125840 0x7f89d4127c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:21.965 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.958+0000 7f89d37fe700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f89d41224c0 0x7f89d4122ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:53874/0 (socket says 172.21.15.170:53874) 2020-03-19T21:18:21.966 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.958+0000 7f89d37fe700 1 -- 172.21.15.170:0/4226437815 learned_addr learned my addr 172.21.15.170:0/4226437815 (peer_addr_for_me v1:172.21.15.170:0/0) 2020-03-19T21:18:21.966 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.958+0000 7f89d2ffd700 1 -- 172.21.15.170:0/4226437815 >> v1:172.21.15.170:6789/0 conn(0x7f89d41224c0 legacy=0x7f89d4122ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:21.966 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.958+0000 7f89d27fc700 1 -- 172.21.15.170:0/4226437815 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2645140560 0 0) 0x7f89c40010d0 con 0x7f89d41224c0 2020-03-19T21:18:21.966 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.959+0000 7f89d2ffd700 1 -- 172.21.15.170:0/4226437815 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89d4090b60 con 0x7f89d4125840 2020-03-19T21:18:21.966 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.959+0000 7f89d2ffd700 1 --2- 172.21.15.170:0/4226437815 >> v2:172.21.15.170:3300/0 conn(0x7f89d4125840 0x7f89d4127c30 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f89c8002f30 tx=0x7f89c8007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:21.966 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.959+0000 7f89d2ffd700 1 --2- 172.21.15.170:0/4226437815 >> v2:172.21.15.170:3300/0 conn(0x7f89d4125840 0x7f89d4127c30 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f89c8002f30 tx=0x7f89c8007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:21.967 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.959+0000 7f89d2ffd700 1 --2- 172.21.15.170:0/4226437815 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89d4125840 0x7f89d4127c30 secure :-1 s=READY pgs=191 cs=0 l=1 rx=0x7f89c8002f30 tx=0x7f89c8007a40).ready entity=mon.0 client_cookie=7e0df0a5922e09f server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:21.967 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.959+0000 7f89d2ffd700 1 --2- 172.21.15.170:0/4226437815 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89d4125840 0x7f89d4127c30 secure :-1 s=THROTTLE_DONE pgs=191 cs=0 l=1 rx=0x7f89c8002f30 tx=0x7f89c8007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:21.967 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.959+0000 7f89d27fc700 1 -- 172.21.15.170:0/4226437815 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f89c8008e80 con 0x7f89d4125840 2020-03-19T21:18:21.967 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.959+0000 7f89d2ffd700 1 --2- 172.21.15.170:0/4226437815 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89d4125840 0x7f89d4127c30 secure :-1 s=THROTTLE_DONE pgs=191 cs=0 l=1 rx=0x7f89c8002f30 tx=0x7f89c8007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:21.967 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.959+0000 7f89d27fc700 1 -- 172.21.15.170:0/4226437815 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89c80113f0 con 0x7f89d4125840 2020-03-19T21:18:21.967 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.959+0000 7f89d2ffd700 1 --2- 172.21.15.170:0/4226437815 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89d4125840 0x7f89d4127c30 secure :-1 s=THROTTLE_DONE pgs=191 cs=0 l=1 rx=0x7f89c8002f30 tx=0x7f89c8007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:21.967 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.959+0000 7f89d27fc700 1 -- 172.21.15.170:0/4226437815 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f89c80116d0 con 0x7f89d4125840 2020-03-19T21:18:21.968 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.959+0000 7f89d9df9700 1 -- 172.21.15.170:0/4226437815 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89d4125840 msgr2=0x7f89d4127c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:21.968 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.959+0000 7f89d9df9700 1 --2- 172.21.15.170:0/4226437815 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89d4125840 0x7f89d4127c30 secure :-1 s=READY pgs=191 cs=0 l=1 rx=0x7f89c8002f30 tx=0x7f89c8007a40).stop 2020-03-19T21:18:21.968 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.959+0000 7f89d9df9700 1 -- 172.21.15.170:0/4226437815 shutdown_connections 2020-03-19T21:18:21.968 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.959+0000 7f89d9df9700 1 --2- 172.21.15.170:0/4226437815 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89d4125840 0x7f89d4127c30 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:21.968 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.959+0000 7f89d9df9700 1 -- 172.21.15.170:0/4226437815 >> 172.21.15.170:0/4226437815 conn(0x7f89d4091390 msgr2=0x7f89d4091770 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:21.968 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.959+0000 7f89d9df9700 1 -- 172.21.15.170:0/4226437815 shutdown_connections 2020-03-19T21:18:21.969 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.959+0000 7f89d9df9700 1 -- 172.21.15.170:0/4226437815 wait complete. 2020-03-19T21:18:21.969 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.960+0000 7f89d9df9700 1 Processor -- start 2020-03-19T21:18:21.969 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.960+0000 7f89d9df9700 1 -- start start 2020-03-19T21:18:21.969 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.961+0000 7f89d9df9700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f89d4125840 0x7f89d408f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:21.969 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.961+0000 7f89d9df9700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89d4066340 con 0x7f89d41224c0 2020-03-19T21:18:21.969 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.961+0000 7f89d9df9700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89d410d170 con 0x7f89d4125840 2020-03-19T21:18:21.970 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.961+0000 7f89d37fe700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f89d41224c0 0x7f89d408f1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:53878/0 (socket says 172.21.15.170:53878) 2020-03-19T21:18:21.970 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.961+0000 7f89d37fe700 1 -- 172.21.15.170:0/1874932266 learned_addr learned my addr 172.21.15.170:0/1874932266 (peer_addr_for_me v1:172.21.15.170:0/0) 2020-03-19T21:18:21.970 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.961+0000 7f89d2ffd700 1 --2- 172.21.15.170:0/1874932266 >> v2:172.21.15.170:3300/0 conn(0x7f89d4125840 0x7f89d408f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:21.970 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.961+0000 7f89d2ffd700 1 -- 172.21.15.170:0/1874932266 >> v1:172.21.15.170:6789/0 conn(0x7f89d41224c0 legacy=0x7f89d408f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:21.970 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.961+0000 7f89d2ffd700 1 -- 172.21.15.170:0/1874932266 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89d410b2d0 con 0x7f89d4125840 2020-03-19T21:18:21.970 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.961+0000 7f89d0ff9700 1 -- 172.21.15.170:0/1874932266 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3634794542 0 0) 0x7f89d4066340 con 0x7f89d41224c0 2020-03-19T21:18:21.971 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.961+0000 7f89d2ffd700 1 --2- 172.21.15.170:0/1874932266 >> v2:172.21.15.170:3300/0 conn(0x7f89d4125840 0x7f89d408f8d0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f89c8002f30 tx=0x7f89c8007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:21.971 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.961+0000 7f89d2ffd700 1 --2- 172.21.15.170:0/1874932266 >> v2:172.21.15.170:3300/0 conn(0x7f89d4125840 0x7f89d408f8d0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f89c8002f30 tx=0x7f89c8007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:21.971 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.961+0000 7f89d2ffd700 1 --2- 172.21.15.170:0/1874932266 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89d4125840 0x7f89d408f8d0 secure :-1 s=READY pgs=193 cs=0 l=1 rx=0x7f89c8002f30 tx=0x7f89c8007a40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:21.971 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.961+0000 7f89d2ffd700 1 --2- 172.21.15.170:0/1874932266 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89d4125840 0x7f89d408f8d0 secure :-1 s=THROTTLE_DONE pgs=193 cs=0 l=1 rx=0x7f89c8002f30 tx=0x7f89c8007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:21.971 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.961+0000 7f89d0ff9700 1 -- 172.21.15.170:0/1874932266 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f89c80116c0 con 0x7f89d4125840 2020-03-19T21:18:21.971 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.961+0000 7f89d2ffd700 1 --2- 172.21.15.170:0/1874932266 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89d4125840 0x7f89d408f8d0 secure :-1 s=THROTTLE_DONE pgs=193 cs=0 l=1 rx=0x7f89c8002f30 tx=0x7f89c8007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:21.971 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.961+0000 7f89d0ff9700 1 -- 172.21.15.170:0/1874932266 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89c800f830 con 0x7f89d4125840 2020-03-19T21:18:21.972 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.961+0000 7f89d2ffd700 1 --2- 172.21.15.170:0/1874932266 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89d4125840 0x7f89d408f8d0 secure :-1 s=THROTTLE_DONE pgs=193 cs=0 l=1 rx=0x7f89c8002f30 tx=0x7f89c8007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:21.972 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.961+0000 7f89d9df9700 1 -- 172.21.15.170:0/1874932266 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89d410c020 con 0x7f89d4125840 2020-03-19T21:18:21.972 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.961+0000 7f89d0ff9700 1 -- 172.21.15.170:0/1874932266 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f89c8007280 con 0x7f89d4125840 2020-03-19T21:18:21.972 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.961+0000 7f89d9df9700 1 -- 172.21.15.170:0/1874932266 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89d408fd30 con 0x7f89d4125840 2020-03-19T21:18:21.972 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.962+0000 7f89d2ffd700 1 --2- 172.21.15.170:0/1874932266 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89d4125840 0x7f89d408f8d0 secure :-1 s=THROTTLE_DONE pgs=193 cs=0 l=1 rx=0x7f89c8002f30 tx=0x7f89c8007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:21.972 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.962+0000 7f89d0ff9700 1 -- 172.21.15.170:0/1874932266 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 4) v1 ==== 29603+0+0 (secure 0 0 0) 0x7f89c8007480 con 0x7f89d4125840 2020-03-19T21:18:21.972 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.962+0000 7f89d2ffd700 1 --2- 172.21.15.170:0/1874932266 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89d4125840 0x7f89d408f8d0 secure :-1 s=THROTTLE_DONE pgs=193 cs=0 l=1 rx=0x7f89c8002f30 tx=0x7f89c8007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:21.973 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.962+0000 7f89d0ff9700 1 --2- 172.21.15.170:0/1874932266 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f89b4028290 0x7f89b402a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:21.973 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.962+0000 7f89d0ff9700 1 -- 172.21.15.170:0/1874932266 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3403+0+0 (secure 0 0 0) 0x7f89c8034770 con 0x7f89d4125840 2020-03-19T21:18:21.973 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.962+0000 7f89d37fe700 1 --2- 172.21.15.170:0/1874932266 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f89b4028290 0x7f89b402a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:21.973 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.962+0000 7f89d37fe700 1 --2- 172.21.15.170:0/1874932266 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f89b4028290 0x7f89b402a750 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f89c4005500 tx=0x7f89c40016e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:21.973 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.962+0000 7f89d9df9700 1 -- 172.21.15.170:0/1874932266 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89d4090b60 con 0x7f89d4125840 2020-03-19T21:18:21.973 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.962+0000 7f89d37fe700 1 --2- 172.21.15.170:0/1874932266 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f89b4028290 0x7f89b402a750 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f89c4005500 tx=0x7f89c40016e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:21.973 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.962+0000 7f89d37fe700 1 --2- 172.21.15.170:0/1874932266 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f89b4028290 0x7f89b402a750 secure :-1 s=READY pgs=48 cs=0 l=1 rx=0x7f89c4005500 tx=0x7f89c40016e0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:21.974 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.966+0000 7f89d2ffd700 1 --2- 172.21.15.170:0/1874932266 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89d4125840 0x7f89d408f8d0 secure :-1 s=THROTTLE_DONE pgs=193 cs=0 l=1 rx=0x7f89c8002f30 tx=0x7f89c8007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:21.974 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:21.966+0000 7f89d0ff9700 1 -- 172.21.15.170:0/1874932266 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+126772 (secure 0 0 0) 0x7f89d4090b60 con 0x7f89d4125840 2020-03-19T21:18:22.104 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.102+0000 7f89d9df9700 1 -- 172.21.15.170:0/1874932266 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f89d4090b60 con 0x7f89d4125840 2020-03-19T21:18:22.104 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.103+0000 7f89d2ffd700 1 --2- 172.21.15.170:0/1874932266 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89d4125840 0x7f89d408f8d0 secure :-1 s=THROTTLE_DONE pgs=193 cs=0 l=1 rx=0x7f89c8002f30 tx=0x7f89c8007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:22.104 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.103+0000 7f89d0ff9700 1 -- 172.21.15.170:0/1874932266 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v16) v1 ==== 100+0+10 (secure 0 0 0) 0x7f89c8014020 con 0x7f89d4125840 2020-03-19T21:18:22.104 INFO:teuthology.orchestra.run.smithi170.stdout:pg_num: 8 2020-03-19T21:18:22.106 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.104+0000 7f89d9df9700 1 -- 172.21.15.170:0/1874932266 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f89b4028290 msgr2=0x7f89b402a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:22.106 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.104+0000 7f89d9df9700 1 --2- 172.21.15.170:0/1874932266 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f89b4028290 0x7f89b402a750 secure :-1 s=READY pgs=48 cs=0 l=1 rx=0x7f89c4005500 tx=0x7f89c40016e0).stop 2020-03-19T21:18:22.106 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.104+0000 7f89d9df9700 1 -- 172.21.15.170:0/1874932266 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89d4125840 msgr2=0x7f89d408f8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:22.106 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.104+0000 7f89d9df9700 1 --2- 172.21.15.170:0/1874932266 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89d4125840 0x7f89d408f8d0 secure :-1 s=READY pgs=193 cs=0 l=1 rx=0x7f89c8002f30 tx=0x7f89c8007a40).stop 2020-03-19T21:18:22.106 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.104+0000 7f89d9df9700 1 -- 172.21.15.170:0/1874932266 shutdown_connections 2020-03-19T21:18:22.107 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.104+0000 7f89d9df9700 1 --2- 172.21.15.170:0/1874932266 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f89b4028290 0x7f89b402a750 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:22.107 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.104+0000 7f89d9df9700 1 --2- 172.21.15.170:0/1874932266 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89d4125840 0x7f89d408f8d0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:22.107 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.104+0000 7f89d9df9700 1 -- 172.21.15.170:0/1874932266 >> 172.21.15.170:0/1874932266 conn(0x7f89d4091390 msgr2=0x7f89d411f220 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:22.107 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.104+0000 7f89d9df9700 1 -- 172.21.15.170:0/1874932266 shutdown_connections 2020-03-19T21:18:22.107 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.104+0000 7f89d9df9700 1 -- 172.21.15.170:0/1874932266 wait complete. 2020-03-19T21:18:22.115 INFO:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-03-19T21:18:22.220 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.219+0000 7f23cb898700 1 Processor -- start 2020-03-19T21:18:22.226 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.220+0000 7f23cb898700 1 -- start start 2020-03-19T21:18:22.227 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.220+0000 7f23cb898700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f23c4125250 0x7f23c4127640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T21:18:22.227 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.220+0000 7f23cb898700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23c408eee0 con 0x7f23c41221d0 2020-03-19T21:18:22.227 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.220+0000 7f23cb898700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23c4091060 con 0x7f23c4125250 2020-03-19T21:18:22.227 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.220+0000 7f23c8e33700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f23c4125250 0x7f23c4127640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:22.228 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.220+0000 7f23c9634700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f23c41221d0 0x7f23c41225b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:53884/0 (socket says 172.21.15.170:53884) 2020-03-19T21:18:22.228 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.220+0000 7f23c8e33700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f23c4125250 0x7f23c4127640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53120/0 (socket says 172.21.15.170:53120) 2020-03-19T21:18:22.228 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.220+0000 7f23c9634700 1 -- 172.21.15.170:0/801729778 learned_addr learned my addr 172.21.15.170:0/801729778 (peer_addr_for_me v1:172.21.15.170:0/0) 2020-03-19T21:18:22.228 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.220+0000 7f23c8e33700 1 -- 172.21.15.170:0/801729778 >> v1:172.21.15.170:6789/0 conn(0x7f23c41221d0 legacy=0x7f23c41225b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:22.228 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.220+0000 7f23affff700 1 -- 172.21.15.170:0/801729778 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2411595000 0 0) 0x7f23c408eee0 con 0x7f23c41221d0 2020-03-19T21:18:22.229 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.220+0000 7f23c8e33700 1 -- 172.21.15.170:0/801729778 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23c4121fd0 con 0x7f23c4125250 2020-03-19T21:18:22.229 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.220+0000 7f23c8e33700 1 --2- 172.21.15.170:0/801729778 >> v2:172.21.15.170:3300/0 conn(0x7f23c4125250 0x7f23c4127640 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f23c0002f30 tx=0x7f23c0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:22.229 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.221+0000 7f23c8e33700 1 --2- 172.21.15.170:0/801729778 >> v2:172.21.15.170:3300/0 conn(0x7f23c4125250 0x7f23c4127640 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f23c0002f30 tx=0x7f23c0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:22.229 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.221+0000 7f23c8e33700 1 --2- 172.21.15.170:0/801729778 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f23c4125250 0x7f23c4127640 secure :-1 s=READY pgs=195 cs=0 l=1 rx=0x7f23c0002f30 tx=0x7f23c0007a40).ready entity=mon.0 client_cookie=3384233a8239d68d server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:22.229 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.221+0000 7f23c8e33700 1 --2- 172.21.15.170:0/801729778 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f23c4125250 0x7f23c4127640 secure :-1 s=THROTTLE_DONE pgs=195 cs=0 l=1 rx=0x7f23c0002f30 tx=0x7f23c0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:22.229 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.221+0000 7f23affff700 1 -- 172.21.15.170:0/801729778 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f23c00113f0 con 0x7f23c4125250 2020-03-19T21:18:22.230 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.221+0000 7f23c8e33700 1 --2- 172.21.15.170:0/801729778 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f23c4125250 0x7f23c4127640 secure :-1 s=THROTTLE_DONE pgs=195 cs=0 l=1 rx=0x7f23c0002f30 tx=0x7f23c0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:22.230 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.221+0000 7f23c8e33700 1 --2- 172.21.15.170:0/801729778 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f23c4125250 0x7f23c4127640 secure :-1 s=THROTTLE_DONE pgs=195 cs=0 l=1 rx=0x7f23c0002f30 tx=0x7f23c0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:22.230 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.221+0000 7f23affff700 1 -- 172.21.15.170:0/801729778 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23c0011550 con 0x7f23c4125250 2020-03-19T21:18:22.230 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.221+0000 7f23affff700 1 -- 172.21.15.170:0/801729778 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f23c0011830 con 0x7f23c4125250 2020-03-19T21:18:22.230 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.221+0000 7f23cb898700 1 -- 172.21.15.170:0/801729778 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f23c4125250 msgr2=0x7f23c4127640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:22.231 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.221+0000 7f23cb898700 1 --2- 172.21.15.170:0/801729778 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f23c4125250 0x7f23c4127640 secure :-1 s=READY pgs=195 cs=0 l=1 rx=0x7f23c0002f30 tx=0x7f23c0007a40).stop 2020-03-19T21:18:22.231 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.221+0000 7f23cb898700 1 -- 172.21.15.170:0/801729778 shutdown_connections 2020-03-19T21:18:22.231 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.221+0000 7f23cb898700 1 --2- 172.21.15.170:0/801729778 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f23c4125250 0x7f23c4127640 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:22.231 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.221+0000 7f23cb898700 1 -- 172.21.15.170:0/801729778 >> 172.21.15.170:0/801729778 conn(0x7f23c4091540 msgr2=0x7f23c4091950 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:22.231 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.221+0000 7f23cb898700 1 -- 172.21.15.170:0/801729778 shutdown_connections 2020-03-19T21:18:22.231 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.221+0000 7f23cb898700 1 -- 172.21.15.170:0/801729778 wait complete. 2020-03-19T21:18:22.232 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.222+0000 7f23cb898700 1 Processor -- start 2020-03-19T21:18:22.232 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.222+0000 7f23cb898700 1 -- start start 2020-03-19T21:18:22.232 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.222+0000 7f23cb898700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f23c41221d0 0x7f23c411a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:22.232 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.222+0000 7f23cb898700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23c408d3a0 con 0x7f23c4125250 2020-03-19T21:18:22.232 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.222+0000 7f23cb898700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23c411da30 con 0x7f23c41221d0 2020-03-19T21:18:22.232 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.222+0000 7f23c9634700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f23c41221d0 0x7f23c411a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:22.233 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.222+0000 7f23c9634700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f23c41221d0 0x7f23c411a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53124/0 (socket says 172.21.15.170:53124) 2020-03-19T21:18:22.233 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.222+0000 7f23c9634700 1 -- 172.21.15.170:0/1835284463 learned_addr learned my addr 172.21.15.170:0/1835284463 (peer_addr_for_me v2:172.21.15.170:0/0) 2020-03-19T21:18:22.233 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.223+0000 7f23c9634700 1 -- 172.21.15.170:0/1835284463 >> v1:172.21.15.170:6789/0 conn(0x7f23c4125250 legacy=0x7f23c411afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:22.233 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.223+0000 7f23b6ffd700 1 -- 172.21.15.170:0/1835284463 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2021555888 0 0) 0x7f23c408d3a0 con 0x7f23c4125250 2020-03-19T21:18:22.233 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.223+0000 7f23c9634700 1 -- 172.21.15.170:0/1835284463 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23c411c110 con 0x7f23c41221d0 2020-03-19T21:18:22.233 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.223+0000 7f23c9634700 1 --2- 172.21.15.170:0/1835284463 >> v2:172.21.15.170:3300/0 conn(0x7f23c41221d0 0x7f23c411a4f0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f23c411df70 tx=0x7f23bc010d10).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:22.234 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.223+0000 7f23c9634700 1 --2- 172.21.15.170:0/1835284463 >> v2:172.21.15.170:3300/0 conn(0x7f23c41221d0 0x7f23c411a4f0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f23c411df70 tx=0x7f23bc010d10).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:22.234 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.223+0000 7f23c9634700 1 --2- 172.21.15.170:0/1835284463 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f23c41221d0 0x7f23c411a4f0 secure :-1 s=READY pgs=197 cs=0 l=1 rx=0x7f23c411df70 tx=0x7f23bc010d10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:22.234 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.223+0000 7f23c9634700 1 --2- 172.21.15.170:0/1835284463 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f23c41221d0 0x7f23c411a4f0 secure :-1 s=THROTTLE_DONE pgs=197 cs=0 l=1 rx=0x7f23c411df70 tx=0x7f23bc010d10).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:22.234 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.223+0000 7f23b6ffd700 1 -- 172.21.15.170:0/1835284463 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f23bc012370 con 0x7f23c41221d0 2020-03-19T21:18:22.234 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.223+0000 7f23c9634700 1 --2- 172.21.15.170:0/1835284463 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f23c41221d0 0x7f23c411a4f0 secure :-1 s=THROTTLE_DONE pgs=197 cs=0 l=1 rx=0x7f23c411df70 tx=0x7f23bc010d10).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:22.234 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.223+0000 7f23b6ffd700 1 -- 172.21.15.170:0/1835284463 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23bc0124d0 con 0x7f23c41221d0 2020-03-19T21:18:22.235 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.223+0000 7f23c9634700 1 --2- 172.21.15.170:0/1835284463 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f23c41221d0 0x7f23c411a4f0 secure :-1 s=THROTTLE_DONE pgs=197 cs=0 l=1 rx=0x7f23c411df70 tx=0x7f23bc010d10).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:22.235 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.223+0000 7f23cb898700 1 -- 172.21.15.170:0/1835284463 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23c411e020 con 0x7f23c41221d0 2020-03-19T21:18:22.235 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.223+0000 7f23b6ffd700 1 -- 172.21.15.170:0/1835284463 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f23bc012770 con 0x7f23c41221d0 2020-03-19T21:18:22.235 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.223+0000 7f23cb898700 1 -- 172.21.15.170:0/1835284463 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23c411b920 con 0x7f23c41221d0 2020-03-19T21:18:22.235 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.223+0000 7f23c9634700 1 --2- 172.21.15.170:0/1835284463 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f23c41221d0 0x7f23c411a4f0 secure :-1 s=THROTTLE_DONE pgs=197 cs=0 l=1 rx=0x7f23c411df70 tx=0x7f23bc010d10).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:22.236 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.224+0000 7f23b6ffd700 1 -- 172.21.15.170:0/1835284463 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 4) v1 ==== 29603+0+0 (secure 0 0 0) 0x7f23bc012970 con 0x7f23c41221d0 2020-03-19T21:18:22.236 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.224+0000 7f23c9634700 1 --2- 172.21.15.170:0/1835284463 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f23c41221d0 0x7f23c411a4f0 secure :-1 s=THROTTLE_DONE pgs=197 cs=0 l=1 rx=0x7f23c411df70 tx=0x7f23bc010d10).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:22.236 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.224+0000 7f23b6ffd700 1 --2- 172.21.15.170:0/1835284463 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f23a0028200 0x7f23a002a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:22.236 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.224+0000 7f23b6ffd700 1 -- 172.21.15.170:0/1835284463 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3403+0+0 (secure 0 0 0) 0x7f23bc0347f0 con 0x7f23c41221d0 2020-03-19T21:18:22.236 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.224+0000 7f23c8e33700 1 --2- 172.21.15.170:0/1835284463 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f23a0028200 0x7f23a002a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:22.237 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.224+0000 7f23cb898700 1 -- 172.21.15.170:0/1835284463 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23c4056db0 con 0x7f23c41221d0 2020-03-19T21:18:22.237 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.224+0000 7f23c8e33700 1 --2- 172.21.15.170:0/1835284463 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f23a0028200 0x7f23a002a6c0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f23c0002ed0 tx=0x7f23c0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:22.237 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.224+0000 7f23c8e33700 1 --2- 172.21.15.170:0/1835284463 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f23a0028200 0x7f23a002a6c0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f23c0002ed0 tx=0x7f23c0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:22.237 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.224+0000 7f23c8e33700 1 --2- 172.21.15.170:0/1835284463 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f23a0028200 0x7f23a002a6c0 secure :-1 s=READY pgs=49 cs=0 l=1 rx=0x7f23c0002ed0 tx=0x7f23c0007a40).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:22.238 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.227+0000 7f23c9634700 1 --2- 172.21.15.170:0/1835284463 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f23c41221d0 0x7f23c411a4f0 secure :-1 s=THROTTLE_DONE pgs=197 cs=0 l=1 rx=0x7f23c411df70 tx=0x7f23bc010d10).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:22.238 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.227+0000 7f23b6ffd700 1 -- 172.21.15.170:0/1835284463 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+126772 (secure 0 0 0) 0x7f23bc011cd0 con 0x7f23c41221d0 2020-03-19T21:18:22.413 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.412+0000 7f23cb898700 1 -- 172.21.15.170:0/1835284463 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f23c406aaf0 con 0x7f23c41221d0 2020-03-19T21:18:22.414 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.412+0000 7f23c9634700 1 --2- 172.21.15.170:0/1835284463 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f23c41221d0 0x7f23c411a4f0 secure :-1 s=THROTTLE_DONE pgs=197 cs=0 l=1 rx=0x7f23c411df70 tx=0x7f23bc010d10).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:22.414 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.412+0000 7f23b6ffd700 1 -- 172.21.15.170:0/1835284463 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+1786 (secure 0 0 0) 0x7f23bc011af0 con 0x7f23c41221d0 2020-03-19T21:18:22.415 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:18:22.415 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-03-19T21:18:13.514585+0000","modified":"2020-03-19T21:18:17.196186+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4202},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4202":{"gid":4202,"name":"a","rank":0,"incarnation":5,"state":"up:active","state_seq":2,"addr":"172.21.15.170:6827/2695458498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":2695458498},{"type":"v1","addr":"172.21.15.170:6827","nonce":2695458498}]},"join_fscid":-1,"export_targets":[],"features":4540138292836696063,"flags":0}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2020-03-19T21:18:22.416 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.414+0000 7f23cb898700 1 -- 172.21.15.170:0/1835284463 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f23a0028200 msgr2=0x7f23a002a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:22.416 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.414+0000 7f23cb898700 1 --2- 172.21.15.170:0/1835284463 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f23a0028200 0x7f23a002a6c0 secure :-1 s=READY pgs=49 cs=0 l=1 rx=0x7f23c0002ed0 tx=0x7f23c0007a40).stop 2020-03-19T21:18:22.417 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.414+0000 7f23cb898700 1 -- 172.21.15.170:0/1835284463 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f23c41221d0 msgr2=0x7f23c411a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:22.417 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.414+0000 7f23cb898700 1 --2- 172.21.15.170:0/1835284463 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f23c41221d0 0x7f23c411a4f0 secure :-1 s=READY pgs=197 cs=0 l=1 rx=0x7f23c411df70 tx=0x7f23bc010d10).stop 2020-03-19T21:18:22.417 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.414+0000 7f23cb898700 1 -- 172.21.15.170:0/1835284463 shutdown_connections 2020-03-19T21:18:22.417 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.414+0000 7f23cb898700 1 --2- 172.21.15.170:0/1835284463 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f23a0028200 0x7f23a002a6c0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:22.417 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.414+0000 7f23cb898700 1 --2- 172.21.15.170:0/1835284463 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f23c41221d0 0x7f23c411a4f0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:22.417 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.414+0000 7f23cb898700 1 -- 172.21.15.170:0/1835284463 >> 172.21.15.170:0/1835284463 conn(0x7f23c4091540 msgr2=0x7f23c411fd60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:22.418 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.414+0000 7f23cb898700 1 -- 172.21.15.170:0/1835284463 shutdown_connections 2020-03-19T21:18:22.418 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.414+0000 7f23cb898700 1 -- 172.21.15.170:0/1835284463 wait complete. 2020-03-19T21:18:22.418 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 6 2020-03-19T21:18:22.526 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {u'session_autoclose': 300, u'balancer': u'', u'up': {u'mds_0': 4202}, u'last_failure_osd_epoch': 0, u'in': [0], u'last_failure': 0, u'max_file_size': 1099511627776, u'explicitly_allowed_features': 0, u'damaged': [], u'tableserver': 0, u'metadata_pool': 2, u'failed': [], u'epoch': 6, u'flags': 18, u'max_mds': 1, u'compat': {u'compat': {}, u'ro_compat': {}, u'incompat': {u'feature_10': u'snaprealm v2', u'feature_8': u'no anchor table', u'feature_9': u'file layout v2', u'feature_2': u'client writeable ranges', u'feature_3': u'default file layouts on dirs', u'feature_1': u'base v0.20', u'feature_6': u'dirfrag is stored in omap', u'feature_4': u'dir inode in separate object', u'feature_5': u'mds uses versioned encoding'}}, u'min_compat_client': u'0 (unknown)', u'data_pools': [3], u'info': {u'gid_4202': {u'export_targets': [], u'name': u'a', u'incarnation': 5, u'state_seq': 2, u'state': u'up:active', u'gid': 4202, u'features': 4540138292836696063, u'rank': 0, u'flags': 0, u'join_fscid': -1, u'addrs': {u'addrvec': [{u'nonce': 2695458498, u'type': u'v2', u'addr': u'172.21.15.170:6826'}, {u'nonce': 2695458498, u'type': u'v1', u'addr': u'172.21.15.170:6827'}]}, u'addr': u'172.21.15.170:6827/2695458498'}}, u'fs_name': u'cephfs', u'created': u'2020-03-19T21:18:13.514585+0000', u'standby_count_wanted': 0, u'enabled': True, u'modified': u'2020-03-19T21:18:17.196186+0000', u'session_timeout': 60, u'stopped': [], u'ever_allowed_features': 0, u'root': 0} 2020-03-19T21:18:22.526 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-03-19T21:18:22.526 INFO:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok status 2020-03-19T21:18:22.645 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2020-03-19T21:18:22.645 INFO:teuthology.orchestra.run.smithi170.stdout: "cluster_fsid": "12d0a33b-6b40-4398-90b4-6b5df1da388e", 2020-03-19T21:18:22.645 INFO:teuthology.orchestra.run.smithi170.stdout: "whoami": 0, 2020-03-19T21:18:22.646 INFO:teuthology.orchestra.run.smithi170.stdout: "id": 4202, 2020-03-19T21:18:22.646 INFO:teuthology.orchestra.run.smithi170.stdout: "want_state": "up:active", 2020-03-19T21:18:22.646 INFO:teuthology.orchestra.run.smithi170.stdout: "state": "up:active", 2020-03-19T21:18:22.646 INFO:teuthology.orchestra.run.smithi170.stdout: "rank_uptime": 6.4483569269999998, 2020-03-19T21:18:22.646 INFO:teuthology.orchestra.run.smithi170.stdout: "mdsmap_epoch": 6, 2020-03-19T21:18:22.647 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_epoch": 16, 2020-03-19T21:18:22.647 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_epoch_barrier": 16, 2020-03-19T21:18:22.647 INFO:teuthology.orchestra.run.smithi170.stdout: "uptime": 6.7780510249999999 2020-03-19T21:18:22.647 INFO:teuthology.orchestra.run.smithi170.stdout:} 2020-03-19T21:18:22.653 INFO:tasks.cephfs.filesystem:_json_asok output: { "cluster_fsid": "12d0a33b-6b40-4398-90b4-6b5df1da388e", "whoami": 0, "id": 4202, "want_state": "up:active", "state": "up:active", "rank_uptime": 6.4483569269999998, "mdsmap_epoch": 6, "osdmap_epoch": 16, "osdmap_epoch_barrier": 16, "uptime": 6.7780510249999999 } 2020-03-19T21:18:22.654 INFO:teuthology.run_tasks:Running task exec... 2020-03-19T21:18:22.668 INFO:teuthology.task.exec:Executing custom commands... 2020-03-19T21:18:22.668 INFO:teuthology.task.exec:Running commands on role client.0 host ubuntu@smithi170.front.sepia.ceph.com 2020-03-19T21:18:22.668 INFO:teuthology.orchestra.run.smithi170:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph osd pool create data_cache 4' 2020-03-19T21:18:22.776 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.772+0000 7f48d89f9700 1 Processor -- start 2020-03-19T21:18:22.776 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.772+0000 7f48d89f9700 1 -- start start 2020-03-19T21:18:22.776 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.773+0000 7f48d89f9700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f48d408d7c0 0x7f48d408dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T21:18:22.777 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.773+0000 7f48d89f9700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48d408ebc0 con 0x7f48d408e000 2020-03-19T21:18:22.777 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.773+0000 7f48d89f9700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48d407eac0 con 0x7f48d408d7c0 2020-03-19T21:18:22.777 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.773+0000 7f48d2d9d700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f48d408d7c0 0x7f48d408dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:22.777 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.773+0000 7f48d259c700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f48d408e000 0x7f48d408e4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:53894/0 (socket says 172.21.15.170:53894) 2020-03-19T21:18:22.778 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.773+0000 7f48d259c700 1 -- 172.21.15.170:0/1592589388 learned_addr learned my addr 172.21.15.170:0/1592589388 (peer_addr_for_me v1:172.21.15.170:0/0) 2020-03-19T21:18:22.778 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.773+0000 7f48d2d9d700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f48d408d7c0 0x7f48d408dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53130/0 (socket says 172.21.15.170:53130) 2020-03-19T21:18:22.778 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.773+0000 7f48d2d9d700 1 -- 172.21.15.170:0/1592589388 >> v1:172.21.15.170:6789/0 conn(0x7f48d408e000 legacy=0x7f48d408e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:22.778 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.773+0000 7f48d1d9b700 1 -- 172.21.15.170:0/1592589388 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1344206885 0 0) 0x7f48d408ebc0 con 0x7f48d408e000 2020-03-19T21:18:22.778 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.773+0000 7f48d2d9d700 1 -- 172.21.15.170:0/1592589388 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48d4090bf0 con 0x7f48d408d7c0 2020-03-19T21:18:22.778 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.773+0000 7f48d2d9d700 1 --2- 172.21.15.170:0/1592589388 >> v2:172.21.15.170:3300/0 conn(0x7f48d408d7c0 0x7f48d408dba0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f48d407de70 tx=0x7f48c40088e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:22.779 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.774+0000 7f48d2d9d700 1 --2- 172.21.15.170:0/1592589388 >> v2:172.21.15.170:3300/0 conn(0x7f48d408d7c0 0x7f48d408dba0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f48d407de70 tx=0x7f48c40088e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:22.779 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.774+0000 7f48d2d9d700 1 --2- 172.21.15.170:0/1592589388 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48d408d7c0 0x7f48d408dba0 secure :-1 s=READY pgs=199 cs=0 l=1 rx=0x7f48d407de70 tx=0x7f48c40088e0).ready entity=mon.0 client_cookie=54130dc38d8e61d9 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:22.779 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.774+0000 7f48d2d9d700 1 --2- 172.21.15.170:0/1592589388 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48d408d7c0 0x7f48d408dba0 secure :-1 s=THROTTLE_DONE pgs=199 cs=0 l=1 rx=0x7f48d407de70 tx=0x7f48c40088e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:22.779 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.774+0000 7f48d1d9b700 1 -- 172.21.15.170:0/1592589388 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f48c40113f0 con 0x7f48d408d7c0 2020-03-19T21:18:22.779 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.774+0000 7f48d2d9d700 1 --2- 172.21.15.170:0/1592589388 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48d408d7c0 0x7f48d408dba0 secure :-1 s=THROTTLE_DONE pgs=199 cs=0 l=1 rx=0x7f48d407de70 tx=0x7f48c40088e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:22.779 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.774+0000 7f48d1d9b700 1 -- 172.21.15.170:0/1592589388 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48c4011550 con 0x7f48d408d7c0 2020-03-19T21:18:22.780 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.774+0000 7f48d2d9d700 1 --2- 172.21.15.170:0/1592589388 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48d408d7c0 0x7f48d408dba0 secure :-1 s=THROTTLE_DONE pgs=199 cs=0 l=1 rx=0x7f48d407de70 tx=0x7f48c40088e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:22.780 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.774+0000 7f48d1d9b700 1 -- 172.21.15.170:0/1592589388 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f48c40117f0 con 0x7f48d408d7c0 2020-03-19T21:18:22.780 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.774+0000 7f48d89f9700 1 -- 172.21.15.170:0/1592589388 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48d408d7c0 msgr2=0x7f48d408dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:22.780 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.774+0000 7f48d89f9700 1 --2- 172.21.15.170:0/1592589388 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48d408d7c0 0x7f48d408dba0 secure :-1 s=READY pgs=199 cs=0 l=1 rx=0x7f48d407de70 tx=0x7f48c40088e0).stop 2020-03-19T21:18:22.780 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.774+0000 7f48d89f9700 1 -- 172.21.15.170:0/1592589388 shutdown_connections 2020-03-19T21:18:22.780 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.774+0000 7f48d89f9700 1 --2- 172.21.15.170:0/1592589388 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48d408d7c0 0x7f48d408dba0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:22.780 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.774+0000 7f48d89f9700 1 -- 172.21.15.170:0/1592589388 >> 172.21.15.170:0/1592589388 conn(0x7f48d4094790 msgr2=0x7f48d4094ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:22.781 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.774+0000 7f48d89f9700 1 -- 172.21.15.170:0/1592589388 shutdown_connections 2020-03-19T21:18:22.781 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.774+0000 7f48d89f9700 1 -- 172.21.15.170:0/1592589388 wait complete. 2020-03-19T21:18:22.781 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.775+0000 7f48d89f9700 1 Processor -- start 2020-03-19T21:18:22.781 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.775+0000 7f48d89f9700 1 -- start start 2020-03-19T21:18:22.785 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.775+0000 7f48d89f9700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f48d408d7c0 0x7f48d40753b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:22.785 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.775+0000 7f48d89f9700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48d4093330 con 0x7f48d408e000 2020-03-19T21:18:22.785 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.775+0000 7f48d89f9700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48d4092a90 con 0x7f48d408d7c0 2020-03-19T21:18:22.785 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.776+0000 7f48d2d9d700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f48d408d7c0 0x7f48d40753b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:22.786 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.776+0000 7f48d259c700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f48d408e000 0x7f48d40729c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:53898/0 (socket says 172.21.15.170:53898) 2020-03-19T21:18:22.786 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.776+0000 7f48d259c700 1 -- 172.21.15.170:0/2384778133 learned_addr learned my addr 172.21.15.170:0/2384778133 (peer_addr_for_me v1:172.21.15.170:0/0) 2020-03-19T21:18:22.786 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.776+0000 7f48c37fe700 1 -- 172.21.15.170:0/2384778133 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4080040682 0 0) 0x7f48d4093330 con 0x7f48d408e000 2020-03-19T21:18:22.786 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.776+0000 7f48c37fe700 1 -- 172.21.15.170:0/2384778133 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f48b8003230 con 0x7f48d408e000 2020-03-19T21:18:22.786 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.776+0000 7f48d2d9d700 1 -- 172.21.15.170:0/2384778133 >> v1:172.21.15.170:6789/0 conn(0x7f48d408e000 legacy=0x7f48d40729c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:22.786 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.776+0000 7f48d2d9d700 1 -- 172.21.15.170:0/2384778133 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48d4074260 con 0x7f48d408d7c0 2020-03-19T21:18:22.787 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.776+0000 7f48d2d9d700 1 --2- 172.21.15.170:0/2384778133 >> v2:172.21.15.170:3300/0 conn(0x7f48d408d7c0 0x7f48d40753b0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f48d407de70 tx=0x7f48d408edf0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:22.787 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.776+0000 7f48d2d9d700 1 --2- 172.21.15.170:0/2384778133 >> v2:172.21.15.170:3300/0 conn(0x7f48d408d7c0 0x7f48d40753b0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f48d407de70 tx=0x7f48d408edf0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:22.787 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.776+0000 7f48d2d9d700 1 --2- 172.21.15.170:0/2384778133 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48d408d7c0 0x7f48d40753b0 secure :-1 s=READY pgs=201 cs=0 l=1 rx=0x7f48d407de70 tx=0x7f48d408edf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:22.787 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.776+0000 7f48d2d9d700 1 --2- 172.21.15.170:0/2384778133 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48d408d7c0 0x7f48d40753b0 secure :-1 s=THROTTLE_DONE pgs=201 cs=0 l=1 rx=0x7f48d407de70 tx=0x7f48d408edf0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:22.787 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.776+0000 7f48c37fe700 1 -- 172.21.15.170:0/2384778133 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f48c400f760 con 0x7f48d408d7c0 2020-03-19T21:18:22.787 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.776+0000 7f48d2d9d700 1 --2- 172.21.15.170:0/2384778133 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48d408d7c0 0x7f48d40753b0 secure :-1 s=THROTTLE_DONE pgs=201 cs=0 l=1 rx=0x7f48d407de70 tx=0x7f48d408edf0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:22.788 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.776+0000 7f48d2d9d700 1 --2- 172.21.15.170:0/2384778133 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48d408d7c0 0x7f48d40753b0 secure :-1 s=THROTTLE_DONE pgs=201 cs=0 l=1 rx=0x7f48d407de70 tx=0x7f48d408edf0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:22.788 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.776+0000 7f48c37fe700 1 -- 172.21.15.170:0/2384778133 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48c40056c0 con 0x7f48d408d7c0 2020-03-19T21:18:22.788 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.776+0000 7f48d89f9700 1 -- 172.21.15.170:0/2384778133 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48d407d920 con 0x7f48d408d7c0 2020-03-19T21:18:22.788 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.776+0000 7f48c37fe700 1 -- 172.21.15.170:0/2384778133 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f48c4005960 con 0x7f48d408d7c0 2020-03-19T21:18:22.788 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.776+0000 7f48d89f9700 1 -- 172.21.15.170:0/2384778133 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48d4074c50 con 0x7f48d408d7c0 2020-03-19T21:18:22.789 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.776+0000 7f48d2d9d700 1 --2- 172.21.15.170:0/2384778133 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48d408d7c0 0x7f48d40753b0 secure :-1 s=THROTTLE_DONE pgs=201 cs=0 l=1 rx=0x7f48d407de70 tx=0x7f48d408edf0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:22.789 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.777+0000 7f48c37fe700 1 -- 172.21.15.170:0/2384778133 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 4) v1 ==== 29603+0+0 (secure 0 0 0) 0x7f48c4005bd0 con 0x7f48d408d7c0 2020-03-19T21:18:22.789 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.777+0000 7f48d2d9d700 1 --2- 172.21.15.170:0/2384778133 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48d408d7c0 0x7f48d40753b0 secure :-1 s=THROTTLE_DONE pgs=201 cs=0 l=1 rx=0x7f48d407de70 tx=0x7f48d408edf0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:22.789 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.777+0000 7f48c37fe700 1 --2- 172.21.15.170:0/2384778133 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f48b80287f0 0x7f48b802acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:22.789 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.777+0000 7f48c37fe700 1 -- 172.21.15.170:0/2384778133 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3403+0+0 (secure 0 0 0) 0x7f48c4034600 con 0x7f48d408d7c0 2020-03-19T21:18:22.789 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.777+0000 7f48d259c700 1 --2- 172.21.15.170:0/2384778133 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f48b80287f0 0x7f48b802acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:22.790 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.777+0000 7f48d259c700 1 --2- 172.21.15.170:0/2384778133 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f48b80287f0 0x7f48b802acb0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f48c80010a0 tx=0x7f48c8001700).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:22.790 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.777+0000 7f48d259c700 1 --2- 172.21.15.170:0/2384778133 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f48b80287f0 0x7f48b802acb0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f48c80010a0 tx=0x7f48c8001700).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:22.790 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.777+0000 7f48d259c700 1 --2- 172.21.15.170:0/2384778133 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f48b80287f0 0x7f48b802acb0 secure :-1 s=READY pgs=50 cs=0 l=1 rx=0x7f48c80010a0 tx=0x7f48c8001700).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:22.790 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.777+0000 7f48d89f9700 1 -- 172.21.15.170:0/2384778133 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48d4056db0 con 0x7f48d408d7c0 2020-03-19T21:18:22.790 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.781+0000 7f48d2d9d700 1 --2- 172.21.15.170:0/2384778133 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48d408d7c0 0x7f48d40753b0 secure :-1 s=THROTTLE_DONE pgs=201 cs=0 l=1 rx=0x7f48d407de70 tx=0x7f48d408edf0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:22.791 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.781+0000 7f48c37fe700 1 -- 172.21.15.170:0/2384778133 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+126772 (secure 0 0 0) 0x7f48c4005e60 con 0x7f48d408d7c0 2020-03-19T21:18:22.923 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:22.919+0000 7f48d89f9700 1 -- 172.21.15.170:0/2384778133 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "data_cache", "pg_num": 4} v 0) v1 -- 0x7f48d406ada0 con 0x7f48d408d7c0 2020-03-19T21:18:23.214 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:23.213+0000 7f48d2d9d700 1 --2- 172.21.15.170:0/2384778133 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48d408d7c0 0x7f48d40753b0 secure :-1 s=THROTTLE_DONE pgs=201 cs=0 l=1 rx=0x7f48d407de70 tx=0x7f48d408edf0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:23.215 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:23.213+0000 7f48c37fe700 1 -- 172.21.15.170:0/2384778133 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "data_cache", "pg_num": 4}]=0 pool 'data_cache' created v17) v1 ==== 123+0+0 (secure 0 0 0) 0x7f48c4009e00 con 0x7f48d408d7c0 2020-03-19T21:18:23.217 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:23.216+0000 7f48d89f9700 1 -- 172.21.15.170:0/2384778133 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f48b80287f0 msgr2=0x7f48b802acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:23.218 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:23.216+0000 7f48d89f9700 1 --2- 172.21.15.170:0/2384778133 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f48b80287f0 0x7f48b802acb0 secure :-1 s=READY pgs=50 cs=0 l=1 rx=0x7f48c80010a0 tx=0x7f48c8001700).stop 2020-03-19T21:18:23.218 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:23.216+0000 7f48d89f9700 1 -- 172.21.15.170:0/2384778133 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48d408d7c0 msgr2=0x7f48d40753b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:23.218 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:23.216+0000 7f48d89f9700 1 --2- 172.21.15.170:0/2384778133 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48d408d7c0 0x7f48d40753b0 secure :-1 s=READY pgs=201 cs=0 l=1 rx=0x7f48d407de70 tx=0x7f48d408edf0).stop 2020-03-19T21:18:23.219 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:23.217+0000 7f48d89f9700 1 -- 172.21.15.170:0/2384778133 shutdown_connections 2020-03-19T21:18:23.219 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:23.217+0000 7f48d89f9700 1 --2- 172.21.15.170:0/2384778133 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f48b80287f0 0x7f48b802acb0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:23.219 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:23.217+0000 7f48d89f9700 1 --2- 172.21.15.170:0/2384778133 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48d408d7c0 0x7f48d40753b0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:23.220 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:23.217+0000 7f48d89f9700 1 -- 172.21.15.170:0/2384778133 >> 172.21.15.170:0/2384778133 conn(0x7f48d4094790 msgr2=0x7f48d4063990 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:23.220 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:23.217+0000 7f48d89f9700 1 -- 172.21.15.170:0/2384778133 shutdown_connections 2020-03-19T21:18:23.220 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:23.217+0000 7f48d89f9700 1 -- 172.21.15.170:0/2384778133 wait complete. 2020-03-19T21:18:23.220 INFO:teuthology.orchestra.run.smithi170.stderr:pool 'data_cache' created 2020-03-19T21:18:23.229 INFO:teuthology.orchestra.run.smithi170:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph osd tier add cephfs_data data_cache' 2020-03-19T21:18:23.338 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:23.334+0000 7f4ede823700 1 Processor -- start 2020-03-19T21:18:23.338 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:23.334+0000 7f4ede823700 1 -- start start 2020-03-19T21:18:23.338 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:23.334+0000 7f4ede823700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f4ed808d7c0 0x7f4ed808dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T21:18:23.338 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:23.334+0000 7f4ede823700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ed808ebc0 con 0x7f4ed808e000 2020-03-19T21:18:23.339 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:23.334+0000 7f4ede823700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ed807eac0 con 0x7f4ed808d7c0 2020-03-19T21:18:23.339 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:23.335+0000 7f4edcdc0700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f4ed808d7c0 0x7f4ed808dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:23.339 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:23.335+0000 7f4edcdc0700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f4ed808d7c0 0x7f4ed808dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53140/0 (socket says 172.21.15.170:53140) 2020-03-19T21:18:23.339 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:23.335+0000 7f4edcdc0700 1 -- 172.21.15.170:0/1030169923 learned_addr learned my addr 172.21.15.170:0/1030169923 (peer_addr_for_me v2:172.21.15.170:0/0) 2020-03-19T21:18:23.339 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:23.335+0000 7f4edcdc0700 1 -- 172.21.15.170:0/1030169923 >> v1:172.21.15.170:6789/0 conn(0x7f4ed808e000 legacy=0x7f4ed808e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:23.339 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:23.335+0000 7f4edcdc0700 1 -- 172.21.15.170:0/1030169923 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ed8090bf0 con 0x7f4ed808d7c0 2020-03-19T21:18:23.339 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:23.335+0000 7f4ed77fe700 1 -- 172.21.15.170:0/1030169923 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 106681361 0 0) 0x7f4ed808ebc0 con 0x7f4ed808e000 2020-03-19T21:18:23.340 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:23.335+0000 7f4edcdc0700 1 --2- 172.21.15.170:0/1030169923 >> v2:172.21.15.170:3300/0 conn(0x7f4ed808d7c0 0x7f4ed808dba0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f4ed807de70 tx=0x7f4ed0008c60).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:23.340 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:23.335+0000 7f4edcdc0700 1 --2- 172.21.15.170:0/1030169923 >> v2:172.21.15.170:3300/0 conn(0x7f4ed808d7c0 0x7f4ed808dba0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f4ed807de70 tx=0x7f4ed0008c60).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:23.340 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:23.335+0000 7f4edcdc0700 1 --2- 172.21.15.170:0/1030169923 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4ed808d7c0 0x7f4ed808dba0 secure :-1 s=READY pgs=203 cs=0 l=1 rx=0x7f4ed807de70 tx=0x7f4ed0008c60).ready entity=mon.0 client_cookie=a84401337048aee server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:23.340 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:23.335+0000 7f4edcdc0700 1 --2- 172.21.15.170:0/1030169923 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4ed808d7c0 0x7f4ed808dba0 secure :-1 s=THROTTLE_DONE pgs=203 cs=0 l=1 rx=0x7f4ed807de70 tx=0x7f4ed0008c60).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:23.340 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:23.335+0000 7f4ed77fe700 1 -- 172.21.15.170:0/1030169923 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f4ed0011d90 con 0x7f4ed808d7c0 2020-03-19T21:18:23.340 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:23.335+0000 7f4edcdc0700 1 --2- 172.21.15.170:0/1030169923 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4ed808d7c0 0x7f4ed808dba0 secure :-1 s=THROTTLE_DONE pgs=203 cs=0 l=1 rx=0x7f4ed807de70 tx=0x7f4ed0008c60).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:23.341 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:23.335+0000 7f4ed77fe700 1 -- 172.21.15.170:0/1030169923 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ed000a160 con 0x7f4ed808d7c0 2020-03-19T21:18:23.341 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:23.335+0000 7f4edcdc0700 1 --2- 172.21.15.170:0/1030169923 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4ed808d7c0 0x7f4ed808dba0 secure :-1 s=THROTTLE_DONE pgs=203 cs=0 l=1 rx=0x7f4ed807de70 tx=0x7f4ed0008c60).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:23.341 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:23.335+0000 7f4ed77fe700 1 -- 172.21.15.170:0/1030169923 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f4ed000a400 con 0x7f4ed808d7c0 2020-03-19T21:18:23.341 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:23.335+0000 7f4ede823700 1 -- 172.21.15.170:0/1030169923 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4ed808d7c0 msgr2=0x7f4ed808dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:23.341 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:23.335+0000 7f4ede823700 1 --2- 172.21.15.170:0/1030169923 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4ed808d7c0 0x7f4ed808dba0 secure :-1 s=READY pgs=203 cs=0 l=1 rx=0x7f4ed807de70 tx=0x7f4ed0008c60).stop 2020-03-19T21:18:23.341 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:23.336+0000 7f4ede823700 1 -- 172.21.15.170:0/1030169923 shutdown_connections 2020-03-19T21:18:23.342 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:23.336+0000 7f4ede823700 1 --2- 172.21.15.170:0/1030169923 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4ed808d7c0 0x7f4ed808dba0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:23.342 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:23.336+0000 7f4ede823700 1 -- 172.21.15.170:0/1030169923 >> 172.21.15.170:0/1030169923 conn(0x7f4ed8094790 msgr2=0x7f4ed8094ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:23.342 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:23.336+0000 7f4ede823700 1 -- 172.21.15.170:0/1030169923 shutdown_connections 2020-03-19T21:18:23.342 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:23.336+0000 7f4ede823700 1 -- 172.21.15.170:0/1030169923 wait complete. 2020-03-19T21:18:23.343 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:23.336+0000 7f4ede823700 1 Processor -- start 2020-03-19T21:18:23.343 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:23.336+0000 7f4ede823700 1 -- start start 2020-03-19T21:18:23.346 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:23.337+0000 7f4ede823700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f4ed808e000 0x7f4ed8074370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:23.346 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:23.337+0000 7f4ede823700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ed8073d10 con 0x7f4ed808d7c0 2020-03-19T21:18:23.347 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:23.337+0000 7f4ede823700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ed80747d0 con 0x7f4ed808e000 2020-03-19T21:18:23.347 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:23.337+0000 7f4edcdc0700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f4ed808d7c0 0x7f4ed8071320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:53908/0 (socket says 172.21.15.170:53908) 2020-03-19T21:18:23.347 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:23.337+0000 7f4edcdc0700 1 -- 172.21.15.170:0/2828733340 learned_addr learned my addr 172.21.15.170:0/2828733340 (peer_addr_for_me v1:172.21.15.170:0/0) 2020-03-19T21:18:23.347 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:23.337+0000 7f4ed7fff700 1 --2- 172.21.15.170:0/2828733340 >> v2:172.21.15.170:3300/0 conn(0x7f4ed808e000 0x7f4ed8074370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:23.347 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:23.337+0000 7f4ed57fa700 1 -- 172.21.15.170:0/2828733340 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3326339832 0 0) 0x7f4ed8073d10 con 0x7f4ed808d7c0 2020-03-19T21:18:23.347 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:23.337+0000 7f4ed57fa700 1 -- 172.21.15.170:0/2828733340 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4eb8003230 con 0x7f4ed808d7c0 2020-03-19T21:18:23.348 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:23.337+0000 7f4ed7fff700 1 -- 172.21.15.170:0/2828733340 >> v1:172.21.15.170:6789/0 conn(0x7f4ed808d7c0 legacy=0x7f4ed8071320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:23.348 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:23.337+0000 7f4ed7fff700 1 -- 172.21.15.170:0/2828733340 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ed8073360 con 0x7f4ed808e000 2020-03-19T21:18:23.348 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:23.338+0000 7f4ed7fff700 1 --2- 172.21.15.170:0/2828733340 >> v2:172.21.15.170:3300/0 conn(0x7f4ed808e000 0x7f4ed8074370 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f4ecc000b80 tx=0x7f4ecc0013a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:23.348 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:23.338+0000 7f4ed7fff700 1 --2- 172.21.15.170:0/2828733340 >> v2:172.21.15.170:3300/0 conn(0x7f4ed808e000 0x7f4ed8074370 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f4ecc000b80 tx=0x7f4ecc0013a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:23.348 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:23.338+0000 7f4ed7fff700 1 --2- 172.21.15.170:0/2828733340 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4ed808e000 0x7f4ed8074370 secure :-1 s=READY pgs=205 cs=0 l=1 rx=0x7f4ecc000b80 tx=0x7f4ecc0013a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:23.348 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:23.338+0000 7f4ed7fff700 1 --2- 172.21.15.170:0/2828733340 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4ed808e000 0x7f4ed8074370 secure :-1 s=THROTTLE_DONE pgs=205 cs=0 l=1 rx=0x7f4ecc000b80 tx=0x7f4ecc0013a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:23.349 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:23.338+0000 7f4ed57fa700 1 -- 172.21.15.170:0/2828733340 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f4ecc001ea0 con 0x7f4ed808e000 2020-03-19T21:18:23.349 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:23.338+0000 7f4ed7fff700 1 --2- 172.21.15.170:0/2828733340 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4ed808e000 0x7f4ed8074370 secure :-1 s=THROTTLE_DONE pgs=205 cs=0 l=1 rx=0x7f4ecc000b80 tx=0x7f4ecc0013a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:23.349 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:23.338+0000 7f4ed7fff700 1 --2- 172.21.15.170:0/2828733340 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4ed808e000 0x7f4ed8074370 secure :-1 s=THROTTLE_DONE pgs=205 cs=0 l=1 rx=0x7f4ecc000b80 tx=0x7f4ecc0013a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:23.349 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:23.338+0000 7f4ed57fa700 1 -- 172.21.15.170:0/2828733340 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ecc00cb60 con 0x7f4ed808e000 2020-03-19T21:18:23.349 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:23.338+0000 7f4ede823700 1 -- 172.21.15.170:0/2828733340 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ed807d8f0 con 0x7f4ed808e000 2020-03-19T21:18:23.349 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:23.338+0000 7f4ed57fa700 1 -- 172.21.15.170:0/2828733340 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f4ecc010e70 con 0x7f4ed808e000 2020-03-19T21:18:23.350 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:23.338+0000 7f4ede823700 1 -- 172.21.15.170:0/2828733340 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ed80724e0 con 0x7f4ed808e000 2020-03-19T21:18:23.350 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:23.338+0000 7f4ed7fff700 1 --2- 172.21.15.170:0/2828733340 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4ed808e000 0x7f4ed8074370 secure :-1 s=THROTTLE_DONE pgs=205 cs=0 l=1 rx=0x7f4ecc000b80 tx=0x7f4ecc0013a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:23.350 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:23.338+0000 7f4ed57fa700 1 -- 172.21.15.170:0/2828733340 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 4) v1 ==== 29603+0+0 (secure 0 0 0) 0x7f4ecc0064e0 con 0x7f4ed808e000 2020-03-19T21:18:23.350 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:23.338+0000 7f4ed7fff700 1 --2- 172.21.15.170:0/2828733340 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4ed808e000 0x7f4ed8074370 secure :-1 s=THROTTLE_DONE pgs=205 cs=0 l=1 rx=0x7f4ecc000b80 tx=0x7f4ecc0013a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:23.350 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:23.339+0000 7f4ed57fa700 1 --2- 172.21.15.170:0/2828733340 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f4eb80287f0 0x7f4eb802acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:23.350 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:23.339+0000 7f4ed57fa700 1 -- 172.21.15.170:0/2828733340 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3752+0+0 (secure 0 0 0) 0x7f4ecc033d20 con 0x7f4ed808e000 2020-03-19T21:18:23.350 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:23.339+0000 7f4edcdc0700 1 --2- 172.21.15.170:0/2828733340 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f4eb80287f0 0x7f4eb802acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:23.351 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:23.339+0000 7f4edcdc0700 1 --2- 172.21.15.170:0/2828733340 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f4eb80287f0 0x7f4eb802acb0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f4ed0007ca0 tx=0x7f4ed808edf0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:23.351 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:23.339+0000 7f4ede823700 1 -- 172.21.15.170:0/2828733340 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ed8073b70 con 0x7f4ed808e000 2020-03-19T21:18:23.351 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:23.339+0000 7f4edcdc0700 1 --2- 172.21.15.170:0/2828733340 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f4eb80287f0 0x7f4eb802acb0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f4ed0007ca0 tx=0x7f4ed808edf0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:23.351 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:23.339+0000 7f4edcdc0700 1 --2- 172.21.15.170:0/2828733340 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f4eb80287f0 0x7f4eb802acb0 secure :-1 s=READY pgs=51 cs=0 l=1 rx=0x7f4ed0007ca0 tx=0x7f4ed808edf0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:23.351 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:23.342+0000 7f4ed7fff700 1 --2- 172.21.15.170:0/2828733340 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4ed808e000 0x7f4ed8074370 secure :-1 s=THROTTLE_DONE pgs=205 cs=0 l=1 rx=0x7f4ecc000b80 tx=0x7f4ecc0013a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:23.351 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:23.342+0000 7f4ed57fa700 1 -- 172.21.15.170:0/2828733340 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+126772 (secure 0 0 0) 0x7f4ed8073b70 con 0x7f4ed808e000 2020-03-19T21:18:23.501 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:23.500+0000 7f4ede823700 1 -- 172.21.15.170:0/2828733340 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd tier add", "pool": "cephfs_data", "tierpool": "data_cache"} v 0) v1 -- 0x7f4ed8073b70 con 0x7f4ed808e000 2020-03-19T21:18:24.218 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:24.217+0000 7f4ed7fff700 1 --2- 172.21.15.170:0/2828733340 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4ed808e000 0x7f4ed8074370 secure :-1 s=THROTTLE_DONE pgs=205 cs=0 l=1 rx=0x7f4ecc000b80 tx=0x7f4ecc0013a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:24.218 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:24.217+0000 7f4ed57fa700 1 -- 172.21.15.170:0/2828733340 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd tier add", "pool": "cephfs_data", "tierpool": "data_cache"}]=0 pool 'data_cache' is now (or already was) a tier of 'cephfs_data' v18) v1 ==== 174+0+0 (secure 0 0 0) 0x7f4ed8073b70 con 0x7f4ed808e000 2020-03-19T21:18:24.221 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:24.219+0000 7f4ede823700 1 -- 172.21.15.170:0/2828733340 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f4eb80287f0 msgr2=0x7f4eb802acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:24.221 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:24.219+0000 7f4ede823700 1 --2- 172.21.15.170:0/2828733340 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f4eb80287f0 0x7f4eb802acb0 secure :-1 s=READY pgs=51 cs=0 l=1 rx=0x7f4ed0007ca0 tx=0x7f4ed808edf0).stop 2020-03-19T21:18:24.221 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:24.219+0000 7f4ede823700 1 -- 172.21.15.170:0/2828733340 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4ed808e000 msgr2=0x7f4ed8074370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:24.222 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:24.219+0000 7f4ede823700 1 --2- 172.21.15.170:0/2828733340 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4ed808e000 0x7f4ed8074370 secure :-1 s=READY pgs=205 cs=0 l=1 rx=0x7f4ecc000b80 tx=0x7f4ecc0013a0).stop 2020-03-19T21:18:24.222 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:24.220+0000 7f4ede823700 1 -- 172.21.15.170:0/2828733340 shutdown_connections 2020-03-19T21:18:24.222 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:24.220+0000 7f4ede823700 1 --2- 172.21.15.170:0/2828733340 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f4eb80287f0 0x7f4eb802acb0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:24.222 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:24.220+0000 7f4ede823700 1 --2- 172.21.15.170:0/2828733340 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4ed808e000 0x7f4ed8074370 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:24.222 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:24.220+0000 7f4ede823700 1 -- 172.21.15.170:0/2828733340 >> 172.21.15.170:0/2828733340 conn(0x7f4ed8094790 msgr2=0x7f4ed8063990 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:24.222 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:24.220+0000 7f4ede823700 1 -- 172.21.15.170:0/2828733340 shutdown_connections 2020-03-19T21:18:24.223 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:24.220+0000 7f4ede823700 1 -- 172.21.15.170:0/2828733340 wait complete. 2020-03-19T21:18:24.223 INFO:teuthology.orchestra.run.smithi170.stderr:pool 'data_cache' is now (or already was) a tier of 'cephfs_data' 2020-03-19T21:18:24.236 INFO:teuthology.orchestra.run.smithi170:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph osd tier cache-mode data_cache writeback' 2020-03-19T21:18:24.353 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:24.346+0000 7f9065570700 1 Processor -- start 2020-03-19T21:18:24.353 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:24.346+0000 7f9065570700 1 -- start start 2020-03-19T21:18:24.353 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:24.346+0000 7f9065570700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f906008d7c0 0x7f906008dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T21:18:24.353 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:24.346+0000 7f9065570700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f906008ebc0 con 0x7f906008e000 2020-03-19T21:18:24.354 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:24.346+0000 7f9065570700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f906007eac0 con 0x7f906008d7c0 2020-03-19T21:18:24.354 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:24.347+0000 7f905f7fe700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f906008d7c0 0x7f906008dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:24.354 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:24.347+0000 7f905f7fe700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f906008d7c0 0x7f906008dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53150/0 (socket says 172.21.15.170:53150) 2020-03-19T21:18:24.354 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:24.347+0000 7f905f7fe700 1 -- 172.21.15.170:0/832549136 learned_addr learned my addr 172.21.15.170:0/832549136 (peer_addr_for_me v2:172.21.15.170:0/0) 2020-03-19T21:18:24.354 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:24.347+0000 7f905effd700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f906008e000 0x7f906008e4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:53914/0 (socket says 172.21.15.170:53914) 2020-03-19T21:18:24.354 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:24.347+0000 7f905f7fe700 1 -- 172.21.15.170:0/832549136 >> v1:172.21.15.170:6789/0 conn(0x7f906008e000 legacy=0x7f906008e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:24.354 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:24.347+0000 7f905f7fe700 1 -- 172.21.15.170:0/832549136 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9060090bf0 con 0x7f906008d7c0 2020-03-19T21:18:24.355 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:24.347+0000 7f905e7fc700 1 -- 172.21.15.170:0/832549136 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2408373745 0 0) 0x7f906008ebc0 con 0x7f906008e000 2020-03-19T21:18:24.355 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:24.347+0000 7f905f7fe700 1 --2- 172.21.15.170:0/832549136 >> v2:172.21.15.170:3300/0 conn(0x7f906008d7c0 0x7f906008dba0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f906007de70 tx=0x7f90500088e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:24.355 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:24.347+0000 7f905f7fe700 1 --2- 172.21.15.170:0/832549136 >> v2:172.21.15.170:3300/0 conn(0x7f906008d7c0 0x7f906008dba0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f906007de70 tx=0x7f90500088e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:24.355 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:24.347+0000 7f905f7fe700 1 --2- 172.21.15.170:0/832549136 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f906008d7c0 0x7f906008dba0 secure :-1 s=READY pgs=207 cs=0 l=1 rx=0x7f906007de70 tx=0x7f90500088e0).ready entity=mon.0 client_cookie=e8ddee38e111c87a server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:24.355 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:24.347+0000 7f905f7fe700 1 --2- 172.21.15.170:0/832549136 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f906008d7c0 0x7f906008dba0 secure :-1 s=THROTTLE_DONE pgs=207 cs=0 l=1 rx=0x7f906007de70 tx=0x7f90500088e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:24.356 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:24.347+0000 7f905e7fc700 1 -- 172.21.15.170:0/832549136 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f90500114b0 con 0x7f906008d7c0 2020-03-19T21:18:24.356 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:24.347+0000 7f905f7fe700 1 --2- 172.21.15.170:0/832549136 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f906008d7c0 0x7f906008dba0 secure :-1 s=THROTTLE_DONE pgs=207 cs=0 l=1 rx=0x7f906007de70 tx=0x7f90500088e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:24.356 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:24.347+0000 7f905e7fc700 1 -- 172.21.15.170:0/832549136 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9050011610 con 0x7f906008d7c0 2020-03-19T21:18:24.356 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:24.347+0000 7f905f7fe700 1 --2- 172.21.15.170:0/832549136 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f906008d7c0 0x7f906008dba0 secure :-1 s=THROTTLE_DONE pgs=207 cs=0 l=1 rx=0x7f906007de70 tx=0x7f90500088e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:24.356 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:24.347+0000 7f905e7fc700 1 -- 172.21.15.170:0/832549136 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f90500118b0 con 0x7f906008d7c0 2020-03-19T21:18:24.356 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:24.347+0000 7f9065570700 1 -- 172.21.15.170:0/832549136 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f906008d7c0 msgr2=0x7f906008dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:24.356 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:24.347+0000 7f9065570700 1 --2- 172.21.15.170:0/832549136 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f906008d7c0 0x7f906008dba0 secure :-1 s=READY pgs=207 cs=0 l=1 rx=0x7f906007de70 tx=0x7f90500088e0).stop 2020-03-19T21:18:24.356 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:24.348+0000 7f9065570700 1 -- 172.21.15.170:0/832549136 shutdown_connections 2020-03-19T21:18:24.357 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:24.348+0000 7f9065570700 1 --2- 172.21.15.170:0/832549136 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f906008d7c0 0x7f906008dba0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:24.357 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:24.348+0000 7f9065570700 1 -- 172.21.15.170:0/832549136 >> 172.21.15.170:0/832549136 conn(0x7f9060094790 msgr2=0x7f9060094ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:24.357 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:24.348+0000 7f9065570700 1 -- 172.21.15.170:0/832549136 shutdown_connections 2020-03-19T21:18:24.357 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:24.348+0000 7f9065570700 1 -- 172.21.15.170:0/832549136 wait complete. 2020-03-19T21:18:24.357 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:24.349+0000 7f9065570700 1 Processor -- start 2020-03-19T21:18:24.357 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:24.349+0000 7f9065570700 1 -- start start 2020-03-19T21:18:24.358 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:24.349+0000 7f9065570700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f906008d7c0 0x7f9060092670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:24.358 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:24.349+0000 7f9065570700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f906007dc10 con 0x7f906008e000 2020-03-19T21:18:24.358 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:24.349+0000 7f9065570700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f906007da30 con 0x7f906008d7c0 2020-03-19T21:18:24.358 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:24.349+0000 7f905f7fe700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f906008d7c0 0x7f9060092670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:24.358 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:24.349+0000 7f905effd700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f906008e000 0x7f90600708a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:53918/0 (socket says 172.21.15.170:53918) 2020-03-19T21:18:24.358 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:24.349+0000 7f905effd700 1 -- 172.21.15.170:0/3375848652 learned_addr learned my addr 172.21.15.170:0/3375848652 (peer_addr_for_me v1:172.21.15.170:0/0) 2020-03-19T21:18:24.359 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:24.349+0000 7f905f7fe700 1 -- 172.21.15.170:0/3375848652 >> v1:172.21.15.170:6789/0 conn(0x7f906008e000 legacy=0x7f90600708a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:24.359 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:24.349+0000 7f9047fff700 1 -- 172.21.15.170:0/3375848652 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2227974991 0 0) 0x7f906007dc10 con 0x7f906008e000 2020-03-19T21:18:24.359 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:24.349+0000 7f905f7fe700 1 -- 172.21.15.170:0/3375848652 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9060077dc0 con 0x7f906008d7c0 2020-03-19T21:18:24.359 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:24.350+0000 7f905f7fe700 1 --2- 172.21.15.170:0/3375848652 >> v2:172.21.15.170:3300/0 conn(0x7f906008d7c0 0x7f9060092670 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f906007de70 tx=0x7f906008edf0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:24.359 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:24.350+0000 7f905f7fe700 1 --2- 172.21.15.170:0/3375848652 >> v2:172.21.15.170:3300/0 conn(0x7f906008d7c0 0x7f9060092670 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f906007de70 tx=0x7f906008edf0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:24.359 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:24.350+0000 7f905f7fe700 1 --2- 172.21.15.170:0/3375848652 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f906008d7c0 0x7f9060092670 secure :-1 s=READY pgs=209 cs=0 l=1 rx=0x7f906007de70 tx=0x7f906008edf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:24.360 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:24.350+0000 7f905f7fe700 1 --2- 172.21.15.170:0/3375848652 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f906008d7c0 0x7f9060092670 secure :-1 s=THROTTLE_DONE pgs=209 cs=0 l=1 rx=0x7f906007de70 tx=0x7f906008edf0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:24.360 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:24.350+0000 7f9047fff700 1 -- 172.21.15.170:0/3375848652 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f905000f8e0 con 0x7f906008d7c0 2020-03-19T21:18:24.360 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:24.350+0000 7f905f7fe700 1 --2- 172.21.15.170:0/3375848652 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f906008d7c0 0x7f9060092670 secure :-1 s=THROTTLE_DONE pgs=209 cs=0 l=1 rx=0x7f906007de70 tx=0x7f906008edf0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:24.360 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:24.350+0000 7f9065570700 1 -- 172.21.15.170:0/3375848652 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9060093190 con 0x7f906008d7c0 2020-03-19T21:18:24.360 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:24.350+0000 7f905f7fe700 1 --2- 172.21.15.170:0/3375848652 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f906008d7c0 0x7f9060092670 secure :-1 s=THROTTLE_DONE pgs=209 cs=0 l=1 rx=0x7f906007de70 tx=0x7f906008edf0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:24.360 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:24.350+0000 7f9047fff700 1 -- 172.21.15.170:0/3375848652 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9050007150 con 0x7f906008d7c0 2020-03-19T21:18:24.361 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:24.350+0000 7f9047fff700 1 -- 172.21.15.170:0/3375848652 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f90500073f0 con 0x7f906008d7c0 2020-03-19T21:18:24.361 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:24.350+0000 7f9065570700 1 -- 172.21.15.170:0/3375848652 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9060071f40 con 0x7f906008d7c0 2020-03-19T21:18:24.361 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:24.350+0000 7f905f7fe700 1 --2- 172.21.15.170:0/3375848652 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f906008d7c0 0x7f9060092670 secure :-1 s=THROTTLE_DONE pgs=209 cs=0 l=1 rx=0x7f906007de70 tx=0x7f906008edf0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:24.361 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:24.350+0000 7f9047fff700 1 -- 172.21.15.170:0/3375848652 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 4) v1 ==== 29603+0+0 (secure 0 0 0) 0x7f90500056c0 con 0x7f906008d7c0 2020-03-19T21:18:24.361 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:24.350+0000 7f905f7fe700 1 --2- 172.21.15.170:0/3375848652 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f906008d7c0 0x7f9060092670 secure :-1 s=THROTTLE_DONE pgs=209 cs=0 l=1 rx=0x7f906007de70 tx=0x7f906008edf0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:24.361 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:24.350+0000 7f9047fff700 1 --2- 172.21.15.170:0/3375848652 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f9040028230 0x7f904002a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:24.361 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:24.351+0000 7f9047fff700 1 -- 172.21.15.170:0/3375848652 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 3760+0+0 (secure 0 0 0) 0x7f9050034840 con 0x7f906008d7c0 2020-03-19T21:18:24.362 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:24.351+0000 7f905effd700 1 --2- 172.21.15.170:0/3375848652 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f9040028230 0x7f904002a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:24.362 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:24.351+0000 7f9065570700 1 -- 172.21.15.170:0/3375848652 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9060076e30 con 0x7f906008d7c0 2020-03-19T21:18:24.362 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:24.351+0000 7f905effd700 1 --2- 172.21.15.170:0/3375848652 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f9040028230 0x7f904002a6f0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f90540012f0 tx=0x7f9054001840).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:24.362 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:24.351+0000 7f905effd700 1 --2- 172.21.15.170:0/3375848652 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f9040028230 0x7f904002a6f0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f90540012f0 tx=0x7f9054001840).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:24.362 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:24.351+0000 7f905effd700 1 --2- 172.21.15.170:0/3375848652 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f9040028230 0x7f904002a6f0 secure :-1 s=READY pgs=52 cs=0 l=1 rx=0x7f90540012f0 tx=0x7f9054001840).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:24.363 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:24.354+0000 7f905f7fe700 1 --2- 172.21.15.170:0/3375848652 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f906008d7c0 0x7f9060092670 secure :-1 s=THROTTLE_DONE pgs=209 cs=0 l=1 rx=0x7f906007de70 tx=0x7f906008edf0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:24.363 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:24.354+0000 7f9047fff700 1 -- 172.21.15.170:0/3375848652 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+126772 (secure 0 0 0) 0x7f9050009de0 con 0x7f906008d7c0 2020-03-19T21:18:24.514 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:24.513+0000 7f9065570700 1 -- 172.21.15.170:0/3375848652 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd tier cache-mode", "pool": "data_cache", "mode": "writeback"} v 0) v1 -- 0x7f9060056db0 con 0x7f906008d7c0 2020-03-19T21:18:25.231 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:25.230+0000 7f905f7fe700 1 --2- 172.21.15.170:0/3375848652 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f906008d7c0 0x7f9060092670 secure :-1 s=THROTTLE_DONE pgs=209 cs=0 l=1 rx=0x7f906007de70 tx=0x7f906008edf0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:25.232 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:25.231+0000 7f9047fff700 1 -- 172.21.15.170:0/3375848652 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd tier cache-mode", "pool": "data_cache", "mode": "writeback"}]=0 set cache-mode for pool 'data_cache' to writeback v19) v1 ==== 159+0+0 (secure 0 0 0) 0x7f905000d920 con 0x7f906008d7c0 2020-03-19T21:18:25.234 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:25.233+0000 7f9065570700 1 -- 172.21.15.170:0/3375848652 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f9040028230 msgr2=0x7f904002a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:25.234 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:25.233+0000 7f9065570700 1 --2- 172.21.15.170:0/3375848652 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f9040028230 0x7f904002a6f0 secure :-1 s=READY pgs=52 cs=0 l=1 rx=0x7f90540012f0 tx=0x7f9054001840).stop 2020-03-19T21:18:25.235 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:25.233+0000 7f9065570700 1 -- 172.21.15.170:0/3375848652 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f906008d7c0 msgr2=0x7f9060092670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:25.235 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:25.233+0000 7f9065570700 1 --2- 172.21.15.170:0/3375848652 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f906008d7c0 0x7f9060092670 secure :-1 s=READY pgs=209 cs=0 l=1 rx=0x7f906007de70 tx=0x7f906008edf0).stop 2020-03-19T21:18:25.235 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:25.233+0000 7f9065570700 1 -- 172.21.15.170:0/3375848652 shutdown_connections 2020-03-19T21:18:25.235 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:25.233+0000 7f9065570700 1 --2- 172.21.15.170:0/3375848652 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f9040028230 0x7f904002a6f0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:25.235 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:25.233+0000 7f9065570700 1 --2- 172.21.15.170:0/3375848652 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f906008d7c0 0x7f9060092670 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:25.235 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:25.233+0000 7f9065570700 1 -- 172.21.15.170:0/3375848652 >> 172.21.15.170:0/3375848652 conn(0x7f9060094790 msgr2=0x7f9060063990 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:25.236 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:25.233+0000 7f9065570700 1 -- 172.21.15.170:0/3375848652 shutdown_connections 2020-03-19T21:18:25.236 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:25.233+0000 7f9065570700 1 -- 172.21.15.170:0/3375848652 wait complete. 2020-03-19T21:18:25.236 INFO:teuthology.orchestra.run.smithi170.stderr:set cache-mode for pool 'data_cache' to writeback 2020-03-19T21:18:25.246 INFO:teuthology.orchestra.run.smithi170:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph osd tier set-overlay cephfs_data data_cache' 2020-03-19T21:18:25.362 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:25.355+0000 7fa0cc132700 1 Processor -- start 2020-03-19T21:18:25.363 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:25.355+0000 7fa0cc132700 1 -- start start 2020-03-19T21:18:25.365 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:25.356+0000 7fa0cc132700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fa0c408e2b0 0x7fa0c408e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T21:18:25.365 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:25.356+0000 7fa0cc132700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0c408ebc0 con 0x7fa0c408d7c0 2020-03-19T21:18:25.365 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:25.356+0000 7fa0cc132700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0c407eac0 con 0x7fa0c408e2b0 2020-03-19T21:18:25.365 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:25.356+0000 7fa0c9ece700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fa0c408e2b0 0x7fa0c408e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:25.365 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:25.356+0000 7fa0c9ece700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fa0c408e2b0 0x7fa0c408e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53158/0 (socket says 172.21.15.170:53158) 2020-03-19T21:18:25.365 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:25.356+0000 7fa0c9ece700 1 -- 172.21.15.170:0/487143375 learned_addr learned my addr 172.21.15.170:0/487143375 (peer_addr_for_me v2:172.21.15.170:0/0) 2020-03-19T21:18:25.366 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:25.356+0000 7fa0ca6cf700 1 --1- 172.21.15.170:0/487143375 >> v1:172.21.15.170:6789/0 conn(0x7fa0c408d7c0 0x7fa0c408dba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:53922/0 (socket says 172.21.15.170:53922) 2020-03-19T21:18:25.366 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:25.356+0000 7fa0c9ece700 1 -- 172.21.15.170:0/487143375 >> v1:172.21.15.170:6789/0 conn(0x7fa0c408d7c0 legacy=0x7fa0c408dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:25.366 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:25.356+0000 7fa0c9ece700 1 -- 172.21.15.170:0/487143375 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0c4090bf0 con 0x7fa0c408e2b0 2020-03-19T21:18:25.366 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:25.356+0000 7fa0c9ece700 1 --2- 172.21.15.170:0/487143375 >> v2:172.21.15.170:3300/0 conn(0x7fa0c408e2b0 0x7fa0c408e760 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7fa0c407de70 tx=0x7fa0b40088e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:25.367 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:25.357+0000 7fa0c9ece700 1 --2- 172.21.15.170:0/487143375 >> v2:172.21.15.170:3300/0 conn(0x7fa0c408e2b0 0x7fa0c408e760 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7fa0c407de70 tx=0x7fa0b40088e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:25.367 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:25.357+0000 7fa0c9ece700 1 --2- 172.21.15.170:0/487143375 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa0c408e2b0 0x7fa0c408e760 secure :-1 s=READY pgs=211 cs=0 l=1 rx=0x7fa0c407de70 tx=0x7fa0b40088e0).ready entity=mon.0 client_cookie=b151303b4d2926c6 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:25.367 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:25.357+0000 7fa0c9ece700 1 --2- 172.21.15.170:0/487143375 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa0c408e2b0 0x7fa0c408e760 secure :-1 s=THROTTLE_DONE pgs=211 cs=0 l=1 rx=0x7fa0c407de70 tx=0x7fa0b40088e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:25.367 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:25.357+0000 7fa0c96cd700 1 -- 172.21.15.170:0/487143375 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fa0b40114b0 con 0x7fa0c408e2b0 2020-03-19T21:18:25.367 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:25.357+0000 7fa0c9ece700 1 --2- 172.21.15.170:0/487143375 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa0c408e2b0 0x7fa0c408e760 secure :-1 s=THROTTLE_DONE pgs=211 cs=0 l=1 rx=0x7fa0c407de70 tx=0x7fa0b40088e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:25.368 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:25.357+0000 7fa0c9ece700 1 --2- 172.21.15.170:0/487143375 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa0c408e2b0 0x7fa0c408e760 secure :-1 s=THROTTLE_DONE pgs=211 cs=0 l=1 rx=0x7fa0c407de70 tx=0x7fa0b40088e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:25.368 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:25.357+0000 7fa0c96cd700 1 -- 172.21.15.170:0/487143375 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0b4011610 con 0x7fa0c408e2b0 2020-03-19T21:18:25.368 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:25.357+0000 7fa0c96cd700 1 -- 172.21.15.170:0/487143375 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fa0b40118b0 con 0x7fa0c408e2b0 2020-03-19T21:18:25.368 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:25.357+0000 7fa0cc132700 1 -- 172.21.15.170:0/487143375 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa0c408e2b0 msgr2=0x7fa0c408e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:25.368 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:25.357+0000 7fa0cc132700 1 --2- 172.21.15.170:0/487143375 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa0c408e2b0 0x7fa0c408e760 secure :-1 s=READY pgs=211 cs=0 l=1 rx=0x7fa0c407de70 tx=0x7fa0b40088e0).stop 2020-03-19T21:18:25.369 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:25.357+0000 7fa0cc132700 1 -- 172.21.15.170:0/487143375 shutdown_connections 2020-03-19T21:18:25.369 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:25.357+0000 7fa0cc132700 1 --2- 172.21.15.170:0/487143375 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa0c408e2b0 0x7fa0c408e760 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:25.369 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:25.357+0000 7fa0cc132700 1 -- 172.21.15.170:0/487143375 >> 172.21.15.170:0/487143375 conn(0x7fa0c4094790 msgr2=0x7fa0c4094ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:25.369 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:25.357+0000 7fa0cc132700 1 -- 172.21.15.170:0/487143375 shutdown_connections 2020-03-19T21:18:25.369 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:25.357+0000 7fa0cc132700 1 -- 172.21.15.170:0/487143375 wait complete. 2020-03-19T21:18:25.370 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:25.358+0000 7fa0cc132700 1 Processor -- start 2020-03-19T21:18:25.370 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:25.358+0000 7fa0cc132700 1 -- start start 2020-03-19T21:18:25.370 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:25.358+0000 7fa0cc132700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fa0c408e2b0 0x7fa0c4073280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:25.370 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:25.358+0000 7fa0cc132700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0c407d980 con 0x7fa0c408d7c0 2020-03-19T21:18:25.370 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:25.359+0000 7fa0cc132700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0c407dbb0 con 0x7fa0c408e2b0 2020-03-19T21:18:25.371 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:25.359+0000 7fa0ca6cf700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fa0c408d7c0 0x7fa0c4072b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:53926/0 (socket says 172.21.15.170:53926) 2020-03-19T21:18:25.371 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:25.359+0000 7fa0c9ece700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fa0c408e2b0 0x7fa0c4073280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:25.371 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:25.359+0000 7fa0ca6cf700 1 -- 172.21.15.170:0/3272955493 learned_addr learned my addr 172.21.15.170:0/3272955493 (peer_addr_for_me v1:172.21.15.170:0/0) 2020-03-19T21:18:25.371 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:25.359+0000 7fa0c9ece700 1 -- 172.21.15.170:0/3272955493 >> v1:172.21.15.170:6789/0 conn(0x7fa0c408d7c0 legacy=0x7fa0c4072b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:25.371 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:25.359+0000 7fa0baffd700 1 -- 172.21.15.170:0/3272955493 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 555190065 0 0) 0x7fa0c407d980 con 0x7fa0c408d7c0 2020-03-19T21:18:25.371 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:25.359+0000 7fa0c9ece700 1 -- 172.21.15.170:0/3272955493 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0c4093c10 con 0x7fa0c408e2b0 2020-03-19T21:18:25.372 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:25.359+0000 7fa0c9ece700 1 --2- 172.21.15.170:0/3272955493 >> v2:172.21.15.170:3300/0 conn(0x7fa0c408e2b0 0x7fa0c4073280 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7fa0c407de70 tx=0x7fa0c408edf0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:25.372 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:25.359+0000 7fa0c9ece700 1 --2- 172.21.15.170:0/3272955493 >> v2:172.21.15.170:3300/0 conn(0x7fa0c408e2b0 0x7fa0c4073280 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7fa0c407de70 tx=0x7fa0c408edf0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:25.372 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:25.359+0000 7fa0c9ece700 1 --2- 172.21.15.170:0/3272955493 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa0c408e2b0 0x7fa0c4073280 secure :-1 s=READY pgs=213 cs=0 l=1 rx=0x7fa0c407de70 tx=0x7fa0c408edf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:25.372 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:25.359+0000 7fa0c9ece700 1 --2- 172.21.15.170:0/3272955493 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa0c408e2b0 0x7fa0c4073280 secure :-1 s=THROTTLE_DONE pgs=213 cs=0 l=1 rx=0x7fa0c407de70 tx=0x7fa0c408edf0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:25.373 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:25.359+0000 7fa0baffd700 1 -- 172.21.15.170:0/3272955493 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fa0b4007150 con 0x7fa0c408e2b0 2020-03-19T21:18:25.373 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:25.359+0000 7fa0c9ece700 1 --2- 172.21.15.170:0/3272955493 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa0c408e2b0 0x7fa0c4073280 secure :-1 s=THROTTLE_DONE pgs=213 cs=0 l=1 rx=0x7fa0c407de70 tx=0x7fa0c408edf0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:25.373 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:25.359+0000 7fa0baffd700 1 -- 172.21.15.170:0/3272955493 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0b40072b0 con 0x7fa0c408e2b0 2020-03-19T21:18:25.373 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:25.359+0000 7fa0c9ece700 1 --2- 172.21.15.170:0/3272955493 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa0c408e2b0 0x7fa0c4073280 secure :-1 s=THROTTLE_DONE pgs=213 cs=0 l=1 rx=0x7fa0c407de70 tx=0x7fa0c408edf0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:25.374 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:25.359+0000 7fa0cc132700 1 -- 172.21.15.170:0/3272955493 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0c4074440 con 0x7fa0c408e2b0 2020-03-19T21:18:25.374 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:25.359+0000 7fa0baffd700 1 -- 172.21.15.170:0/3272955493 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fa0b4007550 con 0x7fa0c408e2b0 2020-03-19T21:18:25.374 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:25.359+0000 7fa0cc132700 1 -- 172.21.15.170:0/3272955493 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0c4093170 con 0x7fa0c408e2b0 2020-03-19T21:18:25.374 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:25.359+0000 7fa0c9ece700 1 --2- 172.21.15.170:0/3272955493 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa0c408e2b0 0x7fa0c4073280 secure :-1 s=THROTTLE_DONE pgs=213 cs=0 l=1 rx=0x7fa0c407de70 tx=0x7fa0c408edf0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:25.374 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:25.360+0000 7fa0baffd700 1 -- 172.21.15.170:0/3272955493 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 4) v1 ==== 29603+0+0 (secure 0 0 0) 0x7fa0b40056c0 con 0x7fa0c408e2b0 2020-03-19T21:18:25.375 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:25.360+0000 7fa0c9ece700 1 --2- 172.21.15.170:0/3272955493 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa0c408e2b0 0x7fa0c4073280 secure :-1 s=THROTTLE_DONE pgs=213 cs=0 l=1 rx=0x7fa0c407de70 tx=0x7fa0c408edf0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:25.375 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:25.360+0000 7fa0baffd700 1 --2- 172.21.15.170:0/3272955493 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7fa0ac028210 0x7fa0ac02a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:25.375 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:25.360+0000 7fa0baffd700 1 -- 172.21.15.170:0/3272955493 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 3760+0+0 (secure 0 0 0) 0x7fa0b4034970 con 0x7fa0c408e2b0 2020-03-19T21:18:25.375 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:25.360+0000 7fa0ca6cf700 1 --2- 172.21.15.170:0/3272955493 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7fa0ac028210 0x7fa0ac02a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:25.375 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:25.360+0000 7fa0ca6cf700 1 --2- 172.21.15.170:0/3272955493 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7fa0ac028210 0x7fa0ac02a6d0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7fa0c0000ef0 tx=0x7fa0c0001740).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:25.376 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:25.360+0000 7fa0ca6cf700 1 --2- 172.21.15.170:0/3272955493 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7fa0ac028210 0x7fa0ac02a6d0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7fa0c0000ef0 tx=0x7fa0c0001740).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:25.376 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:25.360+0000 7fa0ca6cf700 1 --2- 172.21.15.170:0/3272955493 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7fa0ac028210 0x7fa0ac02a6d0 secure :-1 s=READY pgs=53 cs=0 l=1 rx=0x7fa0c0000ef0 tx=0x7fa0c0001740).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:25.376 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:25.360+0000 7fa0cc132700 1 -- 172.21.15.170:0/3272955493 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0c4056db0 con 0x7fa0c408e2b0 2020-03-19T21:18:25.377 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:25.364+0000 7fa0c9ece700 1 --2- 172.21.15.170:0/3272955493 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa0c408e2b0 0x7fa0c4073280 secure :-1 s=THROTTLE_DONE pgs=213 cs=0 l=1 rx=0x7fa0c407de70 tx=0x7fa0c408edf0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:25.377 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:25.364+0000 7fa0baffd700 1 -- 172.21.15.170:0/3272955493 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+126772 (secure 0 0 0) 0x7fa0b4009de0 con 0x7fa0c408e2b0 2020-03-19T21:18:25.525 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:25.524+0000 7fa0cc132700 1 -- 172.21.15.170:0/3272955493 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd tier set-overlay", "pool": "cephfs_data", "overlaypool": "data_cache"} v 0) v1 -- 0x7fa0c406ada0 con 0x7fa0c408e2b0 2020-03-19T21:18:26.232 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:26.231+0000 7fa0c9ece700 1 --2- 172.21.15.170:0/3272955493 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa0c408e2b0 0x7fa0c4073280 secure :-1 s=THROTTLE_DONE pgs=213 cs=0 l=1 rx=0x7fa0c407de70 tx=0x7fa0c408edf0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:26.233 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:26.231+0000 7fa0baffd700 1 -- 172.21.15.170:0/3272955493 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd tier set-overlay", "pool": "cephfs_data", "overlaypool": "data_cache"}]=0 overlay for 'cephfs_data' is now (or already was) 'data_cache' v20) v1 ==== 182+0+0 (secure 0 0 0) 0x7fa0b400d920 con 0x7fa0c408e2b0 2020-03-19T21:18:26.235 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:26.233+0000 7fa0cc132700 1 -- 172.21.15.170:0/3272955493 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7fa0ac028210 msgr2=0x7fa0ac02a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:26.235 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:26.233+0000 7fa0cc132700 1 --2- 172.21.15.170:0/3272955493 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7fa0ac028210 0x7fa0ac02a6d0 secure :-1 s=READY pgs=53 cs=0 l=1 rx=0x7fa0c0000ef0 tx=0x7fa0c0001740).stop 2020-03-19T21:18:26.235 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:26.233+0000 7fa0cc132700 1 -- 172.21.15.170:0/3272955493 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa0c408e2b0 msgr2=0x7fa0c4073280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:26.235 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:26.233+0000 7fa0cc132700 1 --2- 172.21.15.170:0/3272955493 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa0c408e2b0 0x7fa0c4073280 secure :-1 s=READY pgs=213 cs=0 l=1 rx=0x7fa0c407de70 tx=0x7fa0c408edf0).stop 2020-03-19T21:18:26.236 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:26.234+0000 7fa0cc132700 1 -- 172.21.15.170:0/3272955493 shutdown_connections 2020-03-19T21:18:26.236 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:26.234+0000 7fa0cc132700 1 --2- 172.21.15.170:0/3272955493 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7fa0ac028210 0x7fa0ac02a6d0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:26.236 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:26.234+0000 7fa0cc132700 1 --2- 172.21.15.170:0/3272955493 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa0c408e2b0 0x7fa0c4073280 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:26.236 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:26.234+0000 7fa0cc132700 1 -- 172.21.15.170:0/3272955493 >> 172.21.15.170:0/3272955493 conn(0x7fa0c4094790 msgr2=0x7fa0c40637b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:26.236 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:26.234+0000 7fa0cc132700 1 -- 172.21.15.170:0/3272955493 shutdown_connections 2020-03-19T21:18:26.236 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:26.234+0000 7fa0cc132700 1 -- 172.21.15.170:0/3272955493 wait complete. 2020-03-19T21:18:26.237 INFO:teuthology.orchestra.run.smithi170.stderr:overlay for 'cephfs_data' is now (or already was) 'data_cache' 2020-03-19T21:18:26.247 INFO:teuthology.orchestra.run.smithi170:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph osd pool set data_cache hit_set_type bloom' 2020-03-19T21:18:26.362 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:26.358+0000 7f53cb399700 1 Processor -- start 2020-03-19T21:18:26.362 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:26.358+0000 7f53cb399700 1 -- start start 2020-03-19T21:18:26.362 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:26.358+0000 7f53cb399700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f53c407e780 0x7f53c408f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T21:18:26.362 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:26.358+0000 7f53cb399700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53c405c700 con 0x7f53c408f680 2020-03-19T21:18:26.362 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:26.358+0000 7f53cb399700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53c4090520 con 0x7f53c407e780 2020-03-19T21:18:26.363 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:26.359+0000 7f53c9936700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f53c407e780 0x7f53c408f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:26.363 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:26.359+0000 7f53c9135700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f53c408f680 0x7f53c408fb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:53932/0 (socket says 172.21.15.170:53932) 2020-03-19T21:18:26.363 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:26.359+0000 7f53c9135700 1 -- 172.21.15.170:0/4243850208 learned_addr learned my addr 172.21.15.170:0/4243850208 (peer_addr_for_me v1:172.21.15.170:0/0) 2020-03-19T21:18:26.363 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:26.359+0000 7f53c9936700 1 -- 172.21.15.170:0/4243850208 >> v1:172.21.15.170:6789/0 conn(0x7f53c408f680 legacy=0x7f53c408fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:26.363 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:26.359+0000 7f53c9936700 1 -- 172.21.15.170:0/4243850208 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53c407eb80 con 0x7f53c407e780 2020-03-19T21:18:26.363 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:26.359+0000 7f53c8934700 1 -- 172.21.15.170:0/4243850208 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 856360460 0 0) 0x7f53c405c700 con 0x7f53c408f680 2020-03-19T21:18:26.363 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:26.359+0000 7f53c9936700 1 --2- 172.21.15.170:0/4243850208 >> v2:172.21.15.170:3300/0 conn(0x7f53c407e780 0x7f53c408f220 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f53bc0032b0 tx=0x7f53bc007dc0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:26.364 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:26.359+0000 7f53c9936700 1 --2- 172.21.15.170:0/4243850208 >> v2:172.21.15.170:3300/0 conn(0x7f53c407e780 0x7f53c408f220 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f53bc0032b0 tx=0x7f53bc007dc0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:26.364 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:26.359+0000 7f53c9936700 1 --2- 172.21.15.170:0/4243850208 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f53c407e780 0x7f53c408f220 secure :-1 s=READY pgs=215 cs=0 l=1 rx=0x7f53bc0032b0 tx=0x7f53bc007dc0).ready entity=mon.0 client_cookie=8ae16ce57fe544d7 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:26.364 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:26.359+0000 7f53c9936700 1 --2- 172.21.15.170:0/4243850208 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f53c407e780 0x7f53c408f220 secure :-1 s=THROTTLE_DONE pgs=215 cs=0 l=1 rx=0x7f53bc0032b0 tx=0x7f53bc007dc0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:26.364 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:26.359+0000 7f53c8934700 1 -- 172.21.15.170:0/4243850208 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f53bc011d70 con 0x7f53c407e780 2020-03-19T21:18:26.364 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:26.359+0000 7f53c9936700 1 --2- 172.21.15.170:0/4243850208 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f53c407e780 0x7f53c408f220 secure :-1 s=THROTTLE_DONE pgs=215 cs=0 l=1 rx=0x7f53bc0032b0 tx=0x7f53bc007dc0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:26.364 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:26.359+0000 7f53c9936700 1 --2- 172.21.15.170:0/4243850208 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f53c407e780 0x7f53c408f220 secure :-1 s=THROTTLE_DONE pgs=215 cs=0 l=1 rx=0x7f53bc0032b0 tx=0x7f53bc007dc0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:26.364 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:26.359+0000 7f53c8934700 1 -- 172.21.15.170:0/4243850208 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53bc00a160 con 0x7f53c407e780 2020-03-19T21:18:26.364 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:26.359+0000 7f53c8934700 1 -- 172.21.15.170:0/4243850208 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f53bc00a2d0 con 0x7f53c407e780 2020-03-19T21:18:26.365 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:26.359+0000 7f53cb399700 1 -- 172.21.15.170:0/4243850208 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f53c407e780 msgr2=0x7f53c408f220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:26.365 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:26.359+0000 7f53cb399700 1 --2- 172.21.15.170:0/4243850208 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f53c407e780 0x7f53c408f220 secure :-1 s=READY pgs=215 cs=0 l=1 rx=0x7f53bc0032b0 tx=0x7f53bc007dc0).stop 2020-03-19T21:18:26.365 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:26.360+0000 7f53cb399700 1 -- 172.21.15.170:0/4243850208 shutdown_connections 2020-03-19T21:18:26.365 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:26.360+0000 7f53cb399700 1 --2- 172.21.15.170:0/4243850208 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f53c407e780 0x7f53c408f220 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:26.365 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:26.360+0000 7f53cb399700 1 -- 172.21.15.170:0/4243850208 >> 172.21.15.170:0/4243850208 conn(0x7f53c4095460 msgr2=0x7f53c4091320 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:26.366 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:26.360+0000 7f53cb399700 1 -- 172.21.15.170:0/4243850208 shutdown_connections 2020-03-19T21:18:26.366 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:26.360+0000 7f53cb399700 1 -- 172.21.15.170:0/4243850208 wait complete. 2020-03-19T21:18:26.366 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:26.361+0000 7f53cb399700 1 Processor -- start 2020-03-19T21:18:26.366 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:26.361+0000 7f53cb399700 1 -- start start 2020-03-19T21:18:26.370 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:26.361+0000 7f53cb399700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f53c408f680 0x7f53c4074400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:26.371 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:26.361+0000 7f53cb399700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53c4073c70 con 0x7f53c407e780 2020-03-19T21:18:26.371 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:26.361+0000 7f53cb399700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53c4070cb0 con 0x7f53c408f680 2020-03-19T21:18:26.371 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:26.361+0000 7f53c9936700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f53c407e780 0x7f53c40713b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:53936/0 (socket says 172.21.15.170:53936) 2020-03-19T21:18:26.371 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:26.361+0000 7f53c9936700 1 -- 172.21.15.170:0/3165523020 learned_addr learned my addr 172.21.15.170:0/3165523020 (peer_addr_for_me v1:172.21.15.170:0/0) 2020-03-19T21:18:26.371 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:26.361+0000 7f53c9135700 1 --2- 172.21.15.170:0/3165523020 >> v2:172.21.15.170:3300/0 conn(0x7f53c408f680 0x7f53c4074400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:26.372 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:26.361+0000 7f53ba7fc700 1 -- 172.21.15.170:0/3165523020 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3664491243 0 0) 0x7f53c4073c70 con 0x7f53c407e780 2020-03-19T21:18:26.372 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:26.361+0000 7f53ba7fc700 1 -- 172.21.15.170:0/3165523020 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53ac003230 con 0x7f53c407e780 2020-03-19T21:18:26.372 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:26.361+0000 7f53c9135700 1 -- 172.21.15.170:0/3165523020 >> v1:172.21.15.170:6789/0 conn(0x7f53c407e780 legacy=0x7f53c40713b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:26.372 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:26.361+0000 7f53c9135700 1 -- 172.21.15.170:0/3165523020 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53c407d720 con 0x7f53c408f680 2020-03-19T21:18:26.372 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:26.361+0000 7f53c9135700 1 --2- 172.21.15.170:0/3165523020 >> v2:172.21.15.170:3300/0 conn(0x7f53c408f680 0x7f53c4074400 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f53b4000b80 tx=0x7f53b40108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:26.372 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:26.362+0000 7f53c9135700 1 --2- 172.21.15.170:0/3165523020 >> v2:172.21.15.170:3300/0 conn(0x7f53c408f680 0x7f53c4074400 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f53b4000b80 tx=0x7f53b40108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:26.373 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:26.362+0000 7f53c9135700 1 --2- 172.21.15.170:0/3165523020 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f53c408f680 0x7f53c4074400 secure :-1 s=READY pgs=217 cs=0 l=1 rx=0x7f53b4000b80 tx=0x7f53b40108a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:26.373 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:26.362+0000 7f53c9135700 1 --2- 172.21.15.170:0/3165523020 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f53c408f680 0x7f53c4074400 secure :-1 s=THROTTLE_DONE pgs=217 cs=0 l=1 rx=0x7f53b4000b80 tx=0x7f53b40108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:26.373 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:26.362+0000 7f53ba7fc700 1 -- 172.21.15.170:0/3165523020 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f53b4005b20 con 0x7f53c408f680 2020-03-19T21:18:26.373 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:26.362+0000 7f53c9135700 1 --2- 172.21.15.170:0/3165523020 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f53c408f680 0x7f53c4074400 secure :-1 s=THROTTLE_DONE pgs=217 cs=0 l=1 rx=0x7f53b4000b80 tx=0x7f53b40108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:26.373 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:26.362+0000 7f53c9135700 1 --2- 172.21.15.170:0/3165523020 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f53c408f680 0x7f53c4074400 secure :-1 s=THROTTLE_DONE pgs=217 cs=0 l=1 rx=0x7f53b4000b80 tx=0x7f53b40108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:26.374 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:26.362+0000 7f53ba7fc700 1 -- 172.21.15.170:0/3165523020 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53b4005c80 con 0x7f53c408f680 2020-03-19T21:18:26.374 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:26.362+0000 7f53cb399700 1 -- 172.21.15.170:0/3165523020 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53c407d9c0 con 0x7f53c408f680 2020-03-19T21:18:26.374 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:26.362+0000 7f53ba7fc700 1 -- 172.21.15.170:0/3165523020 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f53b40060c0 con 0x7f53c408f680 2020-03-19T21:18:26.374 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:26.362+0000 7f53cb399700 1 -- 172.21.15.170:0/3165523020 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53c40731b0 con 0x7f53c408f680 2020-03-19T21:18:26.374 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:26.362+0000 7f53c9135700 1 --2- 172.21.15.170:0/3165523020 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f53c408f680 0x7f53c4074400 secure :-1 s=THROTTLE_DONE pgs=217 cs=0 l=1 rx=0x7f53b4000b80 tx=0x7f53b40108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:26.374 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:26.362+0000 7f53ba7fc700 1 -- 172.21.15.170:0/3165523020 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 4) v1 ==== 29603+0+0 (secure 0 0 0) 0x7f53b400a320 con 0x7f53c408f680 2020-03-19T21:18:26.374 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:26.362+0000 7f53c9135700 1 --2- 172.21.15.170:0/3165523020 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f53c408f680 0x7f53c4074400 secure :-1 s=THROTTLE_DONE pgs=217 cs=0 l=1 rx=0x7f53b4000b80 tx=0x7f53b40108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:26.375 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:26.362+0000 7f53ba7fc700 1 --2- 172.21.15.170:0/3165523020 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f53ac0287f0 0x7f53ac02acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:26.375 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:26.362+0000 7f53ba7fc700 1 -- 172.21.15.170:0/3165523020 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3760+0+0 (secure 0 0 0) 0x7f53b4034070 con 0x7f53c408f680 2020-03-19T21:18:26.375 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:26.363+0000 7f53c9936700 1 --2- 172.21.15.170:0/3165523020 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f53ac0287f0 0x7f53ac02acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:26.375 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:26.363+0000 7f53c9936700 1 --2- 172.21.15.170:0/3165523020 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f53ac0287f0 0x7f53ac02acb0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f53bc008c60 tx=0x7f53bc007dc0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:26.375 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:26.363+0000 7f53c9936700 1 --2- 172.21.15.170:0/3165523020 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f53ac0287f0 0x7f53ac02acb0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f53bc008c60 tx=0x7f53bc007dc0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:26.375 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:26.363+0000 7f53c9936700 1 --2- 172.21.15.170:0/3165523020 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f53ac0287f0 0x7f53ac02acb0 secure :-1 s=READY pgs=54 cs=0 l=1 rx=0x7f53bc008c60 tx=0x7f53bc007dc0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:26.375 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:26.363+0000 7f53cb399700 1 -- 172.21.15.170:0/3165523020 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53c4073ae0 con 0x7f53c408f680 2020-03-19T21:18:26.376 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:26.366+0000 7f53c9135700 1 --2- 172.21.15.170:0/3165523020 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f53c408f680 0x7f53c4074400 secure :-1 s=THROTTLE_DONE pgs=217 cs=0 l=1 rx=0x7f53b4000b80 tx=0x7f53b40108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:26.376 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:26.366+0000 7f53ba7fc700 1 -- 172.21.15.170:0/3165523020 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+126772 (secure 0 0 0) 0x7f53b4006220 con 0x7f53c408f680 2020-03-19T21:18:26.506 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:26.505+0000 7f53cb399700 1 -- 172.21.15.170:0/3165523020 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "data_cache", "var": "hit_set_type", "val": "bloom"} v 0) v1 -- 0x7f53c4056db0 con 0x7f53c408f680 2020-03-19T21:18:27.238 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:27.237+0000 7f53c9135700 1 --2- 172.21.15.170:0/3165523020 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f53c408f680 0x7f53c4074400 secure :-1 s=THROTTLE_DONE pgs=217 cs=0 l=1 rx=0x7f53b4000b80 tx=0x7f53b40108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:27.238 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:27.237+0000 7f53ba7fc700 1 -- 172.21.15.170:0/3165523020 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "data_cache", "var": "hit_set_type", "val": "bloom"}]=0 set pool 4 hit_set_type to bloom v21) v1 ==== 153+0+0 (secure 0 0 0) 0x7f53b4011d40 con 0x7f53c408f680 2020-03-19T21:18:27.241 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:27.239+0000 7f53cb399700 1 -- 172.21.15.170:0/3165523020 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f53ac0287f0 msgr2=0x7f53ac02acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:27.241 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:27.239+0000 7f53cb399700 1 --2- 172.21.15.170:0/3165523020 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f53ac0287f0 0x7f53ac02acb0 secure :-1 s=READY pgs=54 cs=0 l=1 rx=0x7f53bc008c60 tx=0x7f53bc007dc0).stop 2020-03-19T21:18:27.241 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:27.239+0000 7f53cb399700 1 -- 172.21.15.170:0/3165523020 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f53c408f680 msgr2=0x7f53c4074400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:27.241 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:27.239+0000 7f53cb399700 1 --2- 172.21.15.170:0/3165523020 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f53c408f680 0x7f53c4074400 secure :-1 s=READY pgs=217 cs=0 l=1 rx=0x7f53b4000b80 tx=0x7f53b40108a0).stop 2020-03-19T21:18:27.241 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:27.239+0000 7f53cb399700 1 -- 172.21.15.170:0/3165523020 shutdown_connections 2020-03-19T21:18:27.241 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:27.239+0000 7f53cb399700 1 --2- 172.21.15.170:0/3165523020 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f53ac0287f0 0x7f53ac02acb0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:27.242 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:27.239+0000 7f53cb399700 1 --2- 172.21.15.170:0/3165523020 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f53c408f680 0x7f53c4074400 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:27.242 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:27.239+0000 7f53cb399700 1 -- 172.21.15.170:0/3165523020 >> 172.21.15.170:0/3165523020 conn(0x7f53c4095460 msgr2=0x7f53c40635c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:27.242 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:27.239+0000 7f53cb399700 1 -- 172.21.15.170:0/3165523020 shutdown_connections 2020-03-19T21:18:27.242 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:27.239+0000 7f53cb399700 1 -- 172.21.15.170:0/3165523020 wait complete. 2020-03-19T21:18:27.242 INFO:teuthology.orchestra.run.smithi170.stderr:set pool 4 hit_set_type to bloom 2020-03-19T21:18:27.253 INFO:teuthology.orchestra.run.smithi170:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph osd pool set data_cache hit_set_count 8' 2020-03-19T21:18:27.367 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:27.363+0000 7f85a538c700 1 Processor -- start 2020-03-19T21:18:27.367 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:27.363+0000 7f85a538c700 1 -- start start 2020-03-19T21:18:27.367 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:27.363+0000 7f85a538c700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f85a008f930 0x7f85a008fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T21:18:27.367 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:27.363+0000 7f85a538c700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85a005c700 con 0x7f85a007e780 2020-03-19T21:18:27.368 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:27.363+0000 7f85a538c700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85a0090520 con 0x7f85a008f930 2020-03-19T21:18:27.368 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:27.364+0000 7f859effd700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f85a008f930 0x7f85a008fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:27.368 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:27.364+0000 7f859f7fe700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f85a007e780 0x7f85a008f220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:53942/0 (socket says 172.21.15.170:53942) 2020-03-19T21:18:27.368 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:27.364+0000 7f859f7fe700 1 -- 172.21.15.170:0/165927098 learned_addr learned my addr 172.21.15.170:0/165927098 (peer_addr_for_me v1:172.21.15.170:0/0) 2020-03-19T21:18:27.368 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:27.364+0000 7f859effd700 1 -- 172.21.15.170:0/165927098 >> v1:172.21.15.170:6789/0 conn(0x7f85a007e780 legacy=0x7f85a008f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:27.368 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:27.364+0000 7f859e7fc700 1 -- 172.21.15.170:0/165927098 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1083197068 0 0) 0x7f85a005c700 con 0x7f85a007e780 2020-03-19T21:18:27.368 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:27.364+0000 7f859effd700 1 -- 172.21.15.170:0/165927098 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85a007eb80 con 0x7f85a008f930 2020-03-19T21:18:27.369 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:27.364+0000 7f859effd700 1 --2- 172.21.15.170:0/165927098 >> v2:172.21.15.170:3300/0 conn(0x7f85a008f930 0x7f85a008fdb0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f8594002f30 tx=0x7f8594007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:27.369 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:27.364+0000 7f859effd700 1 --2- 172.21.15.170:0/165927098 >> v2:172.21.15.170:3300/0 conn(0x7f85a008f930 0x7f85a008fdb0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f8594002f30 tx=0x7f8594007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:27.369 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:27.364+0000 7f859effd700 1 --2- 172.21.15.170:0/165927098 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f85a008f930 0x7f85a008fdb0 secure :-1 s=READY pgs=219 cs=0 l=1 rx=0x7f8594002f30 tx=0x7f8594007a40).ready entity=mon.0 client_cookie=4d3035ab7b679bd9 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:27.369 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:27.364+0000 7f859effd700 1 --2- 172.21.15.170:0/165927098 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f85a008f930 0x7f85a008fdb0 secure :-1 s=THROTTLE_DONE pgs=219 cs=0 l=1 rx=0x7f8594002f30 tx=0x7f8594007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:27.369 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:27.364+0000 7f859e7fc700 1 -- 172.21.15.170:0/165927098 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f85940113f0 con 0x7f85a008f930 2020-03-19T21:18:27.369 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:27.364+0000 7f859effd700 1 --2- 172.21.15.170:0/165927098 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f85a008f930 0x7f85a008fdb0 secure :-1 s=THROTTLE_DONE pgs=219 cs=0 l=1 rx=0x7f8594002f30 tx=0x7f8594007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:27.370 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:27.364+0000 7f859e7fc700 1 -- 172.21.15.170:0/165927098 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8594011550 con 0x7f85a008f930 2020-03-19T21:18:27.370 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:27.364+0000 7f859effd700 1 --2- 172.21.15.170:0/165927098 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f85a008f930 0x7f85a008fdb0 secure :-1 s=THROTTLE_DONE pgs=219 cs=0 l=1 rx=0x7f8594002f30 tx=0x7f8594007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:27.370 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:27.364+0000 7f859e7fc700 1 -- 172.21.15.170:0/165927098 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f8594011830 con 0x7f85a008f930 2020-03-19T21:18:27.370 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:27.364+0000 7f85a538c700 1 -- 172.21.15.170:0/165927098 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f85a008f930 msgr2=0x7f85a008fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:27.370 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:27.364+0000 7f85a538c700 1 --2- 172.21.15.170:0/165927098 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f85a008f930 0x7f85a008fdb0 secure :-1 s=READY pgs=219 cs=0 l=1 rx=0x7f8594002f30 tx=0x7f8594007a40).stop 2020-03-19T21:18:27.370 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:27.365+0000 7f85a538c700 1 -- 172.21.15.170:0/165927098 shutdown_connections 2020-03-19T21:18:27.371 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:27.365+0000 7f85a538c700 1 --2- 172.21.15.170:0/165927098 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f85a008f930 0x7f85a008fdb0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:27.371 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:27.365+0000 7f85a538c700 1 -- 172.21.15.170:0/165927098 >> 172.21.15.170:0/165927098 conn(0x7f85a0095460 msgr2=0x7f85a0091320 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:27.371 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:27.365+0000 7f85a538c700 1 -- 172.21.15.170:0/165927098 shutdown_connections 2020-03-19T21:18:27.371 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:27.365+0000 7f85a538c700 1 -- 172.21.15.170:0/165927098 wait complete. 2020-03-19T21:18:27.371 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:27.366+0000 7f85a538c700 1 Processor -- start 2020-03-19T21:18:27.371 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:27.366+0000 7f85a538c700 1 -- start start 2020-03-19T21:18:27.375 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:27.366+0000 7f85a538c700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f85a008f930 0x7f85a0075430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:27.375 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:27.366+0000 7f85a538c700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85a007d930 con 0x7f85a007e780 2020-03-19T21:18:27.375 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:27.366+0000 7f85a538c700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85a0075c10 con 0x7f85a008f930 2020-03-19T21:18:27.375 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:27.366+0000 7f859effd700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f85a008f930 0x7f85a0075430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:27.375 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:27.366+0000 7f859effd700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f85a008f930 0x7f85a0075430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53184/0 (socket says 172.21.15.170:53184) 2020-03-19T21:18:27.375 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:27.366+0000 7f859effd700 1 -- 172.21.15.170:0/1121647174 learned_addr learned my addr 172.21.15.170:0/1121647174 (peer_addr_for_me v2:172.21.15.170:0/0) 2020-03-19T21:18:27.376 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:27.367+0000 7f859effd700 1 -- 172.21.15.170:0/1121647174 >> v1:172.21.15.170:6789/0 conn(0x7f85a007e780 legacy=0x7f85a00729d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:27.376 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:27.367+0000 7f859effd700 1 -- 172.21.15.170:0/1121647174 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85a0075890 con 0x7f85a008f930 2020-03-19T21:18:27.376 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:27.367+0000 7f8587fff700 1 -- 172.21.15.170:0/1121647174 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1143181512 0 0) 0x7f85a007d930 con 0x7f85a007e780 2020-03-19T21:18:27.376 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:27.367+0000 7f859effd700 1 --2- 172.21.15.170:0/1121647174 >> v2:172.21.15.170:3300/0 conn(0x7f85a008f930 0x7f85a0075430 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f8594002f30 tx=0x7f8594007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:27.376 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:27.367+0000 7f859effd700 1 --2- 172.21.15.170:0/1121647174 >> v2:172.21.15.170:3300/0 conn(0x7f85a008f930 0x7f85a0075430 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f8594002f30 tx=0x7f8594007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:27.376 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:27.367+0000 7f859effd700 1 --2- 172.21.15.170:0/1121647174 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f85a008f930 0x7f85a0075430 secure :-1 s=READY pgs=221 cs=0 l=1 rx=0x7f8594002f30 tx=0x7f8594007a40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:27.377 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:27.367+0000 7f859effd700 1 --2- 172.21.15.170:0/1121647174 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f85a008f930 0x7f85a0075430 secure :-1 s=THROTTLE_DONE pgs=221 cs=0 l=1 rx=0x7f8594002f30 tx=0x7f8594007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:27.377 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:27.367+0000 7f8587fff700 1 -- 172.21.15.170:0/1121647174 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f859400f800 con 0x7f85a008f930 2020-03-19T21:18:27.377 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:27.367+0000 7f859effd700 1 --2- 172.21.15.170:0/1121647174 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f85a008f930 0x7f85a0075430 secure :-1 s=THROTTLE_DONE pgs=221 cs=0 l=1 rx=0x7f8594002f30 tx=0x7f8594007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:27.377 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:27.367+0000 7f859effd700 1 --2- 172.21.15.170:0/1121647174 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f85a008f930 0x7f85a0075430 secure :-1 s=THROTTLE_DONE pgs=221 cs=0 l=1 rx=0x7f8594002f30 tx=0x7f8594007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:27.377 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:27.367+0000 7f8587fff700 1 -- 172.21.15.170:0/1121647174 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8594007150 con 0x7f85a008f930 2020-03-19T21:18:27.377 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:27.367+0000 7f85a538c700 1 -- 172.21.15.170:0/1121647174 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85a00743a0 con 0x7f85a008f930 2020-03-19T21:18:27.377 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:27.367+0000 7f8587fff700 1 -- 172.21.15.170:0/1121647174 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f85940073f0 con 0x7f85a008f930 2020-03-19T21:18:27.378 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:27.367+0000 7f85a538c700 1 -- 172.21.15.170:0/1121647174 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85a0073de0 con 0x7f85a008f930 2020-03-19T21:18:27.378 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:27.367+0000 7f859effd700 1 --2- 172.21.15.170:0/1121647174 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f85a008f930 0x7f85a0075430 secure :-1 s=THROTTLE_DONE pgs=221 cs=0 l=1 rx=0x7f8594002f30 tx=0x7f8594007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:27.378 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:27.368+0000 7f8587fff700 1 -- 172.21.15.170:0/1121647174 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 4) v1 ==== 29603+0+0 (secure 0 0 0) 0x7f85940056c0 con 0x7f85a008f930 2020-03-19T21:18:27.378 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:27.368+0000 7f859effd700 1 --2- 172.21.15.170:0/1121647174 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f85a008f930 0x7f85a0075430 secure :-1 s=THROTTLE_DONE pgs=221 cs=0 l=1 rx=0x7f8594002f30 tx=0x7f8594007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:27.378 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:27.368+0000 7f8587fff700 1 --2- 172.21.15.170:0/1121647174 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f8580028260 0x7f858002a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:27.378 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:27.368+0000 7f8587fff700 1 -- 172.21.15.170:0/1121647174 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 3786+0+0 (secure 0 0 0) 0x7f8594034a00 con 0x7f85a008f930 2020-03-19T21:18:27.379 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:27.368+0000 7f859f7fe700 1 --2- 172.21.15.170:0/1121647174 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f8580028260 0x7f858002a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:27.379 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:27.368+0000 7f85a538c700 1 -- 172.21.15.170:0/1121647174 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85a0093b70 con 0x7f85a008f930 2020-03-19T21:18:27.379 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:27.368+0000 7f859f7fe700 1 --2- 172.21.15.170:0/1121647174 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f8580028260 0x7f858002a720 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f8590001110 tx=0x7f8590001680).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:27.379 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:27.368+0000 7f859f7fe700 1 --2- 172.21.15.170:0/1121647174 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f8580028260 0x7f858002a720 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f8590001110 tx=0x7f8590001680).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:27.379 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:27.368+0000 7f859f7fe700 1 --2- 172.21.15.170:0/1121647174 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f8580028260 0x7f858002a720 secure :-1 s=READY pgs=55 cs=0 l=1 rx=0x7f8590001110 tx=0x7f8590001680).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:27.379 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:27.371+0000 7f859effd700 1 --2- 172.21.15.170:0/1121647174 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f85a008f930 0x7f85a0075430 secure :-1 s=THROTTLE_DONE pgs=221 cs=0 l=1 rx=0x7f8594002f30 tx=0x7f8594007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:27.379 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:27.371+0000 7f8587fff700 1 -- 172.21.15.170:0/1121647174 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+126772 (secure 0 0 0) 0x7f85a0093b70 con 0x7f85a008f930 2020-03-19T21:18:27.511 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:27.509+0000 7f85a538c700 1 -- 172.21.15.170:0/1121647174 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "data_cache", "var": "hit_set_count", "val": "8"} v 0) v1 -- 0x7f85a0093b70 con 0x7f85a008f930 2020-03-19T21:18:28.238 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:28.237+0000 7f859effd700 1 --2- 172.21.15.170:0/1121647174 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f85a008f930 0x7f85a0075430 secure :-1 s=THROTTLE_DONE pgs=221 cs=0 l=1 rx=0x7f8594002f30 tx=0x7f8594007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:28.239 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:28.237+0000 7f8587fff700 1 -- 172.21.15.170:0/1121647174 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "data_cache", "var": "hit_set_count", "val": "8"}]=0 set pool 4 hit_set_count to 8 v22) v1 ==== 147+0+0 (secure 0 0 0) 0x7f85a0093b70 con 0x7f85a008f930 2020-03-19T21:18:28.240 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:28.239+0000 7f85a538c700 1 -- 172.21.15.170:0/1121647174 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f8580028260 msgr2=0x7f858002a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:28.241 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:28.239+0000 7f85a538c700 1 --2- 172.21.15.170:0/1121647174 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f8580028260 0x7f858002a720 secure :-1 s=READY pgs=55 cs=0 l=1 rx=0x7f8590001110 tx=0x7f8590001680).stop 2020-03-19T21:18:28.241 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:28.239+0000 7f85a538c700 1 -- 172.21.15.170:0/1121647174 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f85a008f930 msgr2=0x7f85a0075430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:28.241 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:28.239+0000 7f85a538c700 1 --2- 172.21.15.170:0/1121647174 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f85a008f930 0x7f85a0075430 secure :-1 s=READY pgs=221 cs=0 l=1 rx=0x7f8594002f30 tx=0x7f8594007a40).stop 2020-03-19T21:18:28.241 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:28.239+0000 7f85a538c700 1 -- 172.21.15.170:0/1121647174 shutdown_connections 2020-03-19T21:18:28.241 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:28.239+0000 7f85a538c700 1 --2- 172.21.15.170:0/1121647174 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f8580028260 0x7f858002a720 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:28.241 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:28.239+0000 7f85a538c700 1 --2- 172.21.15.170:0/1121647174 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f85a008f930 0x7f85a0075430 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:28.242 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:28.239+0000 7f85a538c700 1 -- 172.21.15.170:0/1121647174 >> 172.21.15.170:0/1121647174 conn(0x7f85a0095460 msgr2=0x7f85a00635c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:28.242 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:28.239+0000 7f85a538c700 1 -- 172.21.15.170:0/1121647174 shutdown_connections 2020-03-19T21:18:28.242 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:28.239+0000 7f85a538c700 1 -- 172.21.15.170:0/1121647174 wait complete. 2020-03-19T21:18:28.242 INFO:teuthology.orchestra.run.smithi170.stderr:set pool 4 hit_set_count to 8 2020-03-19T21:18:28.252 INFO:teuthology.orchestra.run.smithi170:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph osd pool set data_cache hit_set_period 3600' 2020-03-19T21:18:28.364 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:28.360+0000 7fe03db00700 1 Processor -- start 2020-03-19T21:18:28.364 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:28.361+0000 7fe03db00700 1 -- start start 2020-03-19T21:18:28.364 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:28.361+0000 7fe03db00700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fe03808f930 0x7fe03808fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T21:18:28.365 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:28.361+0000 7fe03db00700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe03805c700 con 0x7fe03807e780 2020-03-19T21:18:28.365 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:28.361+0000 7fe03db00700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe038090520 con 0x7fe03808f930 2020-03-19T21:18:28.365 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:28.361+0000 7fe02ffff700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fe03808f930 0x7fe03808fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:28.365 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:28.361+0000 7fe02ffff700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fe03808f930 0x7fe03808fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53188/0 (socket says 172.21.15.170:53188) 2020-03-19T21:18:28.365 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:28.361+0000 7fe02ffff700 1 -- 172.21.15.170:0/787446455 learned_addr learned my addr 172.21.15.170:0/787446455 (peer_addr_for_me v2:172.21.15.170:0/0) 2020-03-19T21:18:28.365 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:28.361+0000 7fe037fff700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fe03807e780 0x7fe03808f220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:53952/0 (socket says 172.21.15.170:53952) 2020-03-19T21:18:28.365 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:28.361+0000 7fe02ffff700 1 -- 172.21.15.170:0/787446455 >> v1:172.21.15.170:6789/0 conn(0x7fe03807e780 legacy=0x7fe03808f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:28.366 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:28.361+0000 7fe0377fe700 1 -- 172.21.15.170:0/787446455 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2268933954 0 0) 0x7fe03805c700 con 0x7fe03807e780 2020-03-19T21:18:28.366 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:28.361+0000 7fe02ffff700 1 -- 172.21.15.170:0/787446455 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe03807eb80 con 0x7fe03808f930 2020-03-19T21:18:28.366 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:28.361+0000 7fe02ffff700 1 --2- 172.21.15.170:0/787446455 >> v2:172.21.15.170:3300/0 conn(0x7fe03808f930 0x7fe03808fdb0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7fe020002f30 tx=0x7fe020007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:28.366 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:28.362+0000 7fe02ffff700 1 --2- 172.21.15.170:0/787446455 >> v2:172.21.15.170:3300/0 conn(0x7fe03808f930 0x7fe03808fdb0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7fe020002f30 tx=0x7fe020007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:28.366 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:28.362+0000 7fe02ffff700 1 --2- 172.21.15.170:0/787446455 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe03808f930 0x7fe03808fdb0 secure :-1 s=READY pgs=223 cs=0 l=1 rx=0x7fe020002f30 tx=0x7fe020007a40).ready entity=mon.0 client_cookie=4e60ea70bffcad03 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:28.366 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:28.362+0000 7fe02ffff700 1 --2- 172.21.15.170:0/787446455 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe03808f930 0x7fe03808fdb0 secure :-1 s=THROTTLE_DONE pgs=223 cs=0 l=1 rx=0x7fe020002f30 tx=0x7fe020007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:28.366 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:28.362+0000 7fe0377fe700 1 -- 172.21.15.170:0/787446455 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fe0200114b0 con 0x7fe03808f930 2020-03-19T21:18:28.367 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:28.362+0000 7fe02ffff700 1 --2- 172.21.15.170:0/787446455 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe03808f930 0x7fe03808fdb0 secure :-1 s=THROTTLE_DONE pgs=223 cs=0 l=1 rx=0x7fe020002f30 tx=0x7fe020007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:28.367 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:28.362+0000 7fe0377fe700 1 -- 172.21.15.170:0/787446455 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe020011610 con 0x7fe03808f930 2020-03-19T21:18:28.367 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:28.362+0000 7fe02ffff700 1 --2- 172.21.15.170:0/787446455 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe03808f930 0x7fe03808fdb0 secure :-1 s=THROTTLE_DONE pgs=223 cs=0 l=1 rx=0x7fe020002f30 tx=0x7fe020007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:28.367 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:28.362+0000 7fe0377fe700 1 -- 172.21.15.170:0/787446455 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fe020011930 con 0x7fe03808f930 2020-03-19T21:18:28.367 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:28.362+0000 7fe03db00700 1 -- 172.21.15.170:0/787446455 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe03808f930 msgr2=0x7fe03808fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:28.367 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:28.362+0000 7fe03db00700 1 --2- 172.21.15.170:0/787446455 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe03808f930 0x7fe03808fdb0 secure :-1 s=READY pgs=223 cs=0 l=1 rx=0x7fe020002f30 tx=0x7fe020007a40).stop 2020-03-19T21:18:28.367 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:28.362+0000 7fe03db00700 1 -- 172.21.15.170:0/787446455 shutdown_connections 2020-03-19T21:18:28.368 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:28.362+0000 7fe03db00700 1 --2- 172.21.15.170:0/787446455 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe03808f930 0x7fe03808fdb0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:28.368 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:28.362+0000 7fe03db00700 1 -- 172.21.15.170:0/787446455 >> 172.21.15.170:0/787446455 conn(0x7fe038095460 msgr2=0x7fe038091320 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:28.368 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:28.362+0000 7fe03db00700 1 -- 172.21.15.170:0/787446455 shutdown_connections 2020-03-19T21:18:28.368 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:28.362+0000 7fe03db00700 1 -- 172.21.15.170:0/787446455 wait complete. 2020-03-19T21:18:28.368 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:28.363+0000 7fe03db00700 1 Processor -- start 2020-03-19T21:18:28.368 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:28.363+0000 7fe03db00700 1 -- start start 2020-03-19T21:18:28.371 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:28.363+0000 7fe03db00700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fe03807e780 0x7fe0380743b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:28.371 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:28.363+0000 7fe03db00700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe038073c20 con 0x7fe03808f930 2020-03-19T21:18:28.372 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:28.363+0000 7fe03db00700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe038070c60 con 0x7fe03807e780 2020-03-19T21:18:28.372 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:28.364+0000 7fe037fff700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fe03807e780 0x7fe0380743b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:28.372 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:28.364+0000 7fe02ffff700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fe03808f930 0x7fe038071360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:53958/0 (socket says 172.21.15.170:53958) 2020-03-19T21:18:28.372 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:28.364+0000 7fe037fff700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fe03807e780 0x7fe0380743b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53192/0 (socket says 172.21.15.170:53192) 2020-03-19T21:18:28.372 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:28.364+0000 7fe02ffff700 1 -- 172.21.15.170:0/3834457708 learned_addr learned my addr 172.21.15.170:0/3834457708 (peer_addr_for_me v1:172.21.15.170:0/0) 2020-03-19T21:18:28.372 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:28.364+0000 7fe037fff700 1 -- 172.21.15.170:0/3834457708 >> v1:172.21.15.170:6789/0 conn(0x7fe03808f930 legacy=0x7fe038071360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:28.373 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:28.364+0000 7fe037fff700 1 -- 172.21.15.170:0/3834457708 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe03807d6d0 con 0x7fe03807e780 2020-03-19T21:18:28.373 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:28.364+0000 7fe0357fa700 1 -- 172.21.15.170:0/3834457708 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3198471384 0 0) 0x7fe038073c20 con 0x7fe03808f930 2020-03-19T21:18:28.373 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:28.364+0000 7fe037fff700 1 --2- 172.21.15.170:0/3834457708 >> v2:172.21.15.170:3300/0 conn(0x7fe03807e780 0x7fe0380743b0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7fe028000b80 tx=0x7fe0280108f0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:28.373 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:28.364+0000 7fe037fff700 1 --2- 172.21.15.170:0/3834457708 >> v2:172.21.15.170:3300/0 conn(0x7fe03807e780 0x7fe0380743b0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7fe028000b80 tx=0x7fe0280108f0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:28.373 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:28.364+0000 7fe037fff700 1 --2- 172.21.15.170:0/3834457708 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe03807e780 0x7fe0380743b0 secure :-1 s=READY pgs=225 cs=0 l=1 rx=0x7fe028000b80 tx=0x7fe0280108f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:28.373 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:28.364+0000 7fe037fff700 1 --2- 172.21.15.170:0/3834457708 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe03807e780 0x7fe0380743b0 secure :-1 s=THROTTLE_DONE pgs=225 cs=0 l=1 rx=0x7fe028000b80 tx=0x7fe0280108f0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:28.373 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:28.364+0000 7fe0357fa700 1 -- 172.21.15.170:0/3834457708 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fe028011480 con 0x7fe03807e780 2020-03-19T21:18:28.374 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:28.364+0000 7fe037fff700 1 --2- 172.21.15.170:0/3834457708 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe03807e780 0x7fe0380743b0 secure :-1 s=THROTTLE_DONE pgs=225 cs=0 l=1 rx=0x7fe028000b80 tx=0x7fe0280108f0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:28.374 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:28.364+0000 7fe037fff700 1 --2- 172.21.15.170:0/3834457708 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe03807e780 0x7fe0380743b0 secure :-1 s=THROTTLE_DONE pgs=225 cs=0 l=1 rx=0x7fe028000b80 tx=0x7fe0280108f0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:28.374 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:28.364+0000 7fe0357fa700 1 -- 172.21.15.170:0/3834457708 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0280115e0 con 0x7fe03807e780 2020-03-19T21:18:28.374 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:28.364+0000 7fe03db00700 1 -- 172.21.15.170:0/3834457708 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe03807d970 con 0x7fe03807e780 2020-03-19T21:18:28.374 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:28.364+0000 7fe0357fa700 1 -- 172.21.15.170:0/3834457708 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fe0280118c0 con 0x7fe03807e780 2020-03-19T21:18:28.374 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:28.364+0000 7fe03db00700 1 -- 172.21.15.170:0/3834457708 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe038073160 con 0x7fe03807e780 2020-03-19T21:18:28.374 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:28.365+0000 7fe037fff700 1 --2- 172.21.15.170:0/3834457708 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe03807e780 0x7fe0380743b0 secure :-1 s=THROTTLE_DONE pgs=225 cs=0 l=1 rx=0x7fe028000b80 tx=0x7fe0280108f0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:28.375 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:28.365+0000 7fe0357fa700 1 -- 172.21.15.170:0/3834457708 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 4) v1 ==== 29603+0+0 (secure 0 0 0) 0x7fe02800b0a0 con 0x7fe03807e780 2020-03-19T21:18:28.375 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:28.365+0000 7fe037fff700 1 --2- 172.21.15.170:0/3834457708 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe03807e780 0x7fe0380743b0 secure :-1 s=THROTTLE_DONE pgs=225 cs=0 l=1 rx=0x7fe028000b80 tx=0x7fe0280108f0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:28.375 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:28.365+0000 7fe0357fa700 1 --2- 172.21.15.170:0/3834457708 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7fe01c028280 0x7fe01c02a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:28.375 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:28.365+0000 7fe0357fa700 1 -- 172.21.15.170:0/3834457708 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 3786+0+0 (secure 0 0 0) 0x7fe028034030 con 0x7fe03807e780 2020-03-19T21:18:28.375 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:28.365+0000 7fe02ffff700 1 --2- 172.21.15.170:0/3834457708 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7fe01c028280 0x7fe01c02a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:28.375 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:28.365+0000 7fe03db00700 1 -- 172.21.15.170:0/3834457708 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe038073a90 con 0x7fe03807e780 2020-03-19T21:18:28.375 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:28.365+0000 7fe02ffff700 1 --2- 172.21.15.170:0/3834457708 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7fe01c028280 0x7fe01c02a740 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7fe020010280 tx=0x7fe020007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:28.376 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:28.365+0000 7fe02ffff700 1 --2- 172.21.15.170:0/3834457708 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7fe01c028280 0x7fe01c02a740 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7fe020010280 tx=0x7fe020007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:28.376 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:28.365+0000 7fe02ffff700 1 --2- 172.21.15.170:0/3834457708 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7fe01c028280 0x7fe01c02a740 secure :-1 s=READY pgs=56 cs=0 l=1 rx=0x7fe020010280 tx=0x7fe020007a40).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:28.376 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:28.369+0000 7fe037fff700 1 --2- 172.21.15.170:0/3834457708 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe03807e780 0x7fe0380743b0 secure :-1 s=THROTTLE_DONE pgs=225 cs=0 l=1 rx=0x7fe028000b80 tx=0x7fe0280108f0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:28.376 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:28.369+0000 7fe0357fa700 1 -- 172.21.15.170:0/3834457708 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+126772 (secure 0 0 0) 0x7fe028011a20 con 0x7fe03807e780 2020-03-19T21:18:28.512 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:28.511+0000 7fe03db00700 1 -- 172.21.15.170:0/3834457708 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "data_cache", "var": "hit_set_period", "val": "3600"} v 0) v1 -- 0x7fe038056db0 con 0x7fe03807e780 2020-03-19T21:18:29.246 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:29.245+0000 7fe037fff700 1 --2- 172.21.15.170:0/3834457708 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe03807e780 0x7fe0380743b0 secure :-1 s=THROTTLE_DONE pgs=225 cs=0 l=1 rx=0x7fe028000b80 tx=0x7fe0280108f0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:29.246 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:29.245+0000 7fe0357fa700 1 -- 172.21.15.170:0/3834457708 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "data_cache", "var": "hit_set_period", "val": "3600"}]=0 set pool 4 hit_set_period to 3600 v23) v1 ==== 155+0+0 (secure 0 0 0) 0x7fe028007160 con 0x7fe03807e780 2020-03-19T21:18:29.248 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:29.247+0000 7fe03db00700 1 -- 172.21.15.170:0/3834457708 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7fe01c028280 msgr2=0x7fe01c02a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:29.248 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:29.247+0000 7fe03db00700 1 --2- 172.21.15.170:0/3834457708 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7fe01c028280 0x7fe01c02a740 secure :-1 s=READY pgs=56 cs=0 l=1 rx=0x7fe020010280 tx=0x7fe020007a40).stop 2020-03-19T21:18:29.249 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:29.247+0000 7fe03db00700 1 -- 172.21.15.170:0/3834457708 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe03807e780 msgr2=0x7fe0380743b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:29.249 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:29.247+0000 7fe03db00700 1 --2- 172.21.15.170:0/3834457708 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe03807e780 0x7fe0380743b0 secure :-1 s=READY pgs=225 cs=0 l=1 rx=0x7fe028000b80 tx=0x7fe0280108f0).stop 2020-03-19T21:18:29.249 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:29.247+0000 7fe03db00700 1 -- 172.21.15.170:0/3834457708 shutdown_connections 2020-03-19T21:18:29.249 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:29.247+0000 7fe03db00700 1 --2- 172.21.15.170:0/3834457708 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7fe01c028280 0x7fe01c02a740 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:29.249 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:29.247+0000 7fe03db00700 1 --2- 172.21.15.170:0/3834457708 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe03807e780 0x7fe0380743b0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:29.249 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:29.247+0000 7fe03db00700 1 -- 172.21.15.170:0/3834457708 >> 172.21.15.170:0/3834457708 conn(0x7fe038095460 msgr2=0x7fe0380635c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:29.249 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:29.247+0000 7fe03db00700 1 -- 172.21.15.170:0/3834457708 shutdown_connections 2020-03-19T21:18:29.250 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:29.247+0000 7fe03db00700 1 -- 172.21.15.170:0/3834457708 wait complete. 2020-03-19T21:18:29.250 INFO:teuthology.orchestra.run.smithi170.stderr:set pool 4 hit_set_period to 3600 2020-03-19T21:18:29.259 INFO:teuthology.orchestra.run.smithi170:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph osd pool set data_cache min_read_recency_for_promote 0' 2020-03-19T21:18:29.371 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:29.367+0000 7fe9e67b8700 1 Processor -- start 2020-03-19T21:18:29.371 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:29.367+0000 7fe9e67b8700 1 -- start start 2020-03-19T21:18:29.371 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:29.368+0000 7fe9e67b8700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fe9e008f930 0x7fe9e008fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T21:18:29.372 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:29.368+0000 7fe9e67b8700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9e005c700 con 0x7fe9e007e780 2020-03-19T21:18:29.372 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:29.368+0000 7fe9e67b8700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9e0090520 con 0x7fe9e008f930 2020-03-19T21:18:29.372 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:29.368+0000 7fe9dffff700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fe9e008f930 0x7fe9e008fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:29.372 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:29.368+0000 7fe9dffff700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fe9e008f930 0x7fe9e008fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53198/0 (socket says 172.21.15.170:53198) 2020-03-19T21:18:29.372 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:29.368+0000 7fe9e4d55700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fe9e007e780 0x7fe9e008f220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:53962/0 (socket says 172.21.15.170:53962) 2020-03-19T21:18:29.372 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:29.368+0000 7fe9dffff700 1 -- 172.21.15.170:0/1717822993 learned_addr learned my addr 172.21.15.170:0/1717822993 (peer_addr_for_me v2:172.21.15.170:0/0) 2020-03-19T21:18:29.372 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:29.368+0000 7fe9dffff700 1 -- 172.21.15.170:0/1717822993 >> v1:172.21.15.170:6789/0 conn(0x7fe9e007e780 legacy=0x7fe9e008f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:29.373 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:29.368+0000 7fe9df7fe700 1 -- 172.21.15.170:0/1717822993 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1943853518 0 0) 0x7fe9e005c700 con 0x7fe9e007e780 2020-03-19T21:18:29.373 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:29.368+0000 7fe9dffff700 1 -- 172.21.15.170:0/1717822993 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9e007eb80 con 0x7fe9e008f930 2020-03-19T21:18:29.373 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:29.368+0000 7fe9dffff700 1 --2- 172.21.15.170:0/1717822993 >> v2:172.21.15.170:3300/0 conn(0x7fe9e008f930 0x7fe9e008fdb0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7fe9d4002f30 tx=0x7fe9d4007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:29.373 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:29.368+0000 7fe9dffff700 1 --2- 172.21.15.170:0/1717822993 >> v2:172.21.15.170:3300/0 conn(0x7fe9e008f930 0x7fe9e008fdb0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7fe9d4002f30 tx=0x7fe9d4007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:29.373 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:29.368+0000 7fe9dffff700 1 --2- 172.21.15.170:0/1717822993 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe9e008f930 0x7fe9e008fdb0 secure :-1 s=READY pgs=227 cs=0 l=1 rx=0x7fe9d4002f30 tx=0x7fe9d4007a40).ready entity=mon.0 client_cookie=6423e43b502d215 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:29.373 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:29.368+0000 7fe9dffff700 1 --2- 172.21.15.170:0/1717822993 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe9e008f930 0x7fe9e008fdb0 secure :-1 s=THROTTLE_DONE pgs=227 cs=0 l=1 rx=0x7fe9d4002f30 tx=0x7fe9d4007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:29.373 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:29.368+0000 7fe9df7fe700 1 -- 172.21.15.170:0/1717822993 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fe9d40114b0 con 0x7fe9e008f930 2020-03-19T21:18:29.374 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:29.368+0000 7fe9dffff700 1 --2- 172.21.15.170:0/1717822993 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe9e008f930 0x7fe9e008fdb0 secure :-1 s=THROTTLE_DONE pgs=227 cs=0 l=1 rx=0x7fe9d4002f30 tx=0x7fe9d4007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:29.374 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:29.368+0000 7fe9df7fe700 1 -- 172.21.15.170:0/1717822993 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9d4011610 con 0x7fe9e008f930 2020-03-19T21:18:29.374 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:29.368+0000 7fe9dffff700 1 --2- 172.21.15.170:0/1717822993 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe9e008f930 0x7fe9e008fdb0 secure :-1 s=THROTTLE_DONE pgs=227 cs=0 l=1 rx=0x7fe9d4002f30 tx=0x7fe9d4007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:29.374 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:29.368+0000 7fe9df7fe700 1 -- 172.21.15.170:0/1717822993 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fe9d4011930 con 0x7fe9e008f930 2020-03-19T21:18:29.374 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:29.368+0000 7fe9e67b8700 1 -- 172.21.15.170:0/1717822993 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe9e008f930 msgr2=0x7fe9e008fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:29.374 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:29.368+0000 7fe9e67b8700 1 --2- 172.21.15.170:0/1717822993 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe9e008f930 0x7fe9e008fdb0 secure :-1 s=READY pgs=227 cs=0 l=1 rx=0x7fe9d4002f30 tx=0x7fe9d4007a40).stop 2020-03-19T21:18:29.374 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:29.369+0000 7fe9e67b8700 1 -- 172.21.15.170:0/1717822993 shutdown_connections 2020-03-19T21:18:29.375 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:29.369+0000 7fe9e67b8700 1 --2- 172.21.15.170:0/1717822993 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe9e008f930 0x7fe9e008fdb0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:29.375 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:29.369+0000 7fe9e67b8700 1 -- 172.21.15.170:0/1717822993 >> 172.21.15.170:0/1717822993 conn(0x7fe9e0095460 msgr2=0x7fe9e0091320 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:29.375 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:29.369+0000 7fe9e67b8700 1 -- 172.21.15.170:0/1717822993 shutdown_connections 2020-03-19T21:18:29.375 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:29.369+0000 7fe9e67b8700 1 -- 172.21.15.170:0/1717822993 wait complete. 2020-03-19T21:18:29.375 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:29.370+0000 7fe9e67b8700 1 Processor -- start 2020-03-19T21:18:29.375 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:29.370+0000 7fe9e67b8700 1 -- start start 2020-03-19T21:18:29.378 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:29.370+0000 7fe9e67b8700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fe9e008f930 0x7fe9e00743b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:29.379 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:29.370+0000 7fe9e67b8700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9e0073c20 con 0x7fe9e007e780 2020-03-19T21:18:29.379 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:29.370+0000 7fe9e67b8700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9e0070c60 con 0x7fe9e008f930 2020-03-19T21:18:29.379 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:29.370+0000 7fe9e4d55700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fe9e007e780 0x7fe9e0071360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:53966/0 (socket says 172.21.15.170:53966) 2020-03-19T21:18:29.379 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:29.370+0000 7fe9e4d55700 1 -- 172.21.15.170:0/1788822413 learned_addr learned my addr 172.21.15.170:0/1788822413 (peer_addr_for_me v1:172.21.15.170:0/0) 2020-03-19T21:18:29.379 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:29.370+0000 7fe9dffff700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fe9e008f930 0x7fe9e00743b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:29.379 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:29.371+0000 7fe9dffff700 1 -- 172.21.15.170:0/1788822413 >> v1:172.21.15.170:6789/0 conn(0x7fe9e007e780 legacy=0x7fe9e0071360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:29.379 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:29.371+0000 7fe9dffff700 1 -- 172.21.15.170:0/1788822413 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9e007d6d0 con 0x7fe9e008f930 2020-03-19T21:18:29.380 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:29.371+0000 7fe9dd7fa700 1 -- 172.21.15.170:0/1788822413 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2983406057 0 0) 0x7fe9e0073c20 con 0x7fe9e007e780 2020-03-19T21:18:29.380 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:29.371+0000 7fe9dffff700 1 --2- 172.21.15.170:0/1788822413 >> v2:172.21.15.170:3300/0 conn(0x7fe9e008f930 0x7fe9e00743b0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7fe9d4002f30 tx=0x7fe9d4007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:29.380 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:29.371+0000 7fe9dffff700 1 --2- 172.21.15.170:0/1788822413 >> v2:172.21.15.170:3300/0 conn(0x7fe9e008f930 0x7fe9e00743b0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7fe9d4002f30 tx=0x7fe9d4007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:29.380 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:29.371+0000 7fe9dffff700 1 --2- 172.21.15.170:0/1788822413 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe9e008f930 0x7fe9e00743b0 secure :-1 s=READY pgs=229 cs=0 l=1 rx=0x7fe9d4002f30 tx=0x7fe9d4007a40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:29.380 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:29.371+0000 7fe9dffff700 1 --2- 172.21.15.170:0/1788822413 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe9e008f930 0x7fe9e00743b0 secure :-1 s=THROTTLE_DONE pgs=229 cs=0 l=1 rx=0x7fe9d4002f30 tx=0x7fe9d4007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:29.380 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:29.371+0000 7fe9dd7fa700 1 -- 172.21.15.170:0/1788822413 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fe9d4011990 con 0x7fe9e008f930 2020-03-19T21:18:29.380 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:29.371+0000 7fe9dffff700 1 --2- 172.21.15.170:0/1788822413 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe9e008f930 0x7fe9e00743b0 secure :-1 s=THROTTLE_DONE pgs=229 cs=0 l=1 rx=0x7fe9d4002f30 tx=0x7fe9d4007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:29.381 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:29.371+0000 7fe9dffff700 1 --2- 172.21.15.170:0/1788822413 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe9e008f930 0x7fe9e00743b0 secure :-1 s=THROTTLE_DONE pgs=229 cs=0 l=1 rx=0x7fe9d4002f30 tx=0x7fe9d4007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:29.381 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:29.371+0000 7fe9dd7fa700 1 -- 172.21.15.170:0/1788822413 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9d4007150 con 0x7fe9e008f930 2020-03-19T21:18:29.381 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:29.371+0000 7fe9e67b8700 1 -- 172.21.15.170:0/1788822413 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9e007d970 con 0x7fe9e008f930 2020-03-19T21:18:29.381 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:29.371+0000 7fe9dd7fa700 1 -- 172.21.15.170:0/1788822413 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fe9d40073f0 con 0x7fe9e008f930 2020-03-19T21:18:29.381 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:29.371+0000 7fe9e67b8700 1 -- 172.21.15.170:0/1788822413 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9e0073160 con 0x7fe9e008f930 2020-03-19T21:18:29.381 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:29.371+0000 7fe9dffff700 1 --2- 172.21.15.170:0/1788822413 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe9e008f930 0x7fe9e00743b0 secure :-1 s=THROTTLE_DONE pgs=229 cs=0 l=1 rx=0x7fe9d4002f30 tx=0x7fe9d4007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:29.381 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:29.372+0000 7fe9dd7fa700 1 -- 172.21.15.170:0/1788822413 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 4) v1 ==== 29603+0+0 (secure 0 0 0) 0x7fe9d40056c0 con 0x7fe9e008f930 2020-03-19T21:18:29.382 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:29.372+0000 7fe9dffff700 1 --2- 172.21.15.170:0/1788822413 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe9e008f930 0x7fe9e00743b0 secure :-1 s=THROTTLE_DONE pgs=229 cs=0 l=1 rx=0x7fe9d4002f30 tx=0x7fe9d4007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:29.382 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:29.372+0000 7fe9dd7fa700 1 --2- 172.21.15.170:0/1788822413 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7fe9c0028280 0x7fe9c002a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:29.382 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:29.372+0000 7fe9dd7fa700 1 -- 172.21.15.170:0/1788822413 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 3786+0+0 (secure 0 0 0) 0x7fe9d40348c0 con 0x7fe9e008f930 2020-03-19T21:18:29.382 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:29.372+0000 7fe9e4d55700 1 --2- 172.21.15.170:0/1788822413 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7fe9c0028280 0x7fe9c002a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:29.382 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:29.372+0000 7fe9e67b8700 1 -- 172.21.15.170:0/1788822413 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9e0073a90 con 0x7fe9e008f930 2020-03-19T21:18:29.382 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:29.372+0000 7fe9e4d55700 1 --2- 172.21.15.170:0/1788822413 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7fe9c0028280 0x7fe9c002a740 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7fe9d0001170 tx=0x7fe9d0001780).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:29.382 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:29.372+0000 7fe9e4d55700 1 --2- 172.21.15.170:0/1788822413 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7fe9c0028280 0x7fe9c002a740 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7fe9d0001170 tx=0x7fe9d0001780).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:29.383 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:29.372+0000 7fe9e4d55700 1 --2- 172.21.15.170:0/1788822413 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7fe9c0028280 0x7fe9c002a740 secure :-1 s=READY pgs=57 cs=0 l=1 rx=0x7fe9d0001170 tx=0x7fe9d0001780).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:29.383 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:29.375+0000 7fe9dffff700 1 --2- 172.21.15.170:0/1788822413 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe9e008f930 0x7fe9e00743b0 secure :-1 s=THROTTLE_DONE pgs=229 cs=0 l=1 rx=0x7fe9d4002f30 tx=0x7fe9d4007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:29.383 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:29.375+0000 7fe9dd7fa700 1 -- 172.21.15.170:0/1788822413 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+126772 (secure 0 0 0) 0x7fe9d4009de0 con 0x7fe9e008f930 2020-03-19T21:18:29.515 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:29.514+0000 7fe9e67b8700 1 -- 172.21.15.170:0/1788822413 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "data_cache", "var": "min_read_recency_for_promote", "val": "0"} v 0) v1 -- 0x7fe9e0056db0 con 0x7fe9e008f930 2020-03-19T21:18:30.250 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.249+0000 7fe9dffff700 1 --2- 172.21.15.170:0/1788822413 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe9e008f930 0x7fe9e00743b0 secure :-1 s=THROTTLE_DONE pgs=229 cs=0 l=1 rx=0x7fe9d4002f30 tx=0x7fe9d4007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:30.250 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.249+0000 7fe9dd7fa700 1 -- 172.21.15.170:0/1788822413 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "data_cache", "var": "min_read_recency_for_promote", "val": "0"}]=0 set pool 4 min_read_recency_for_promote to 0 v24) v1 ==== 177+0+0 (secure 0 0 0) 0x7fe9d400d400 con 0x7fe9e008f930 2020-03-19T21:18:30.252 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.251+0000 7fe9e67b8700 1 -- 172.21.15.170:0/1788822413 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7fe9c0028280 msgr2=0x7fe9c002a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:30.253 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.251+0000 7fe9e67b8700 1 --2- 172.21.15.170:0/1788822413 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7fe9c0028280 0x7fe9c002a740 secure :-1 s=READY pgs=57 cs=0 l=1 rx=0x7fe9d0001170 tx=0x7fe9d0001780).stop 2020-03-19T21:18:30.253 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.251+0000 7fe9e67b8700 1 -- 172.21.15.170:0/1788822413 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe9e008f930 msgr2=0x7fe9e00743b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:30.253 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.251+0000 7fe9e67b8700 1 --2- 172.21.15.170:0/1788822413 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe9e008f930 0x7fe9e00743b0 secure :-1 s=READY pgs=229 cs=0 l=1 rx=0x7fe9d4002f30 tx=0x7fe9d4007a40).stop 2020-03-19T21:18:30.253 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.251+0000 7fe9e67b8700 1 -- 172.21.15.170:0/1788822413 shutdown_connections 2020-03-19T21:18:30.253 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.251+0000 7fe9e67b8700 1 --2- 172.21.15.170:0/1788822413 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7fe9c0028280 0x7fe9c002a740 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:30.253 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.251+0000 7fe9e67b8700 1 --2- 172.21.15.170:0/1788822413 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe9e008f930 0x7fe9e00743b0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:30.254 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.251+0000 7fe9e67b8700 1 -- 172.21.15.170:0/1788822413 >> 172.21.15.170:0/1788822413 conn(0x7fe9e0095460 msgr2=0x7fe9e00635c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:30.254 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.251+0000 7fe9e67b8700 1 -- 172.21.15.170:0/1788822413 shutdown_connections 2020-03-19T21:18:30.254 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.251+0000 7fe9e67b8700 1 -- 172.21.15.170:0/1788822413 wait complete. 2020-03-19T21:18:30.254 INFO:teuthology.orchestra.run.smithi170.stderr:set pool 4 min_read_recency_for_promote to 0 2020-03-19T21:18:30.265 INFO:teuthology.run_tasks:Running task ceph-fuse... 2020-03-19T21:18:30.304 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2020-03-19T21:18:30.304 INFO:tasks.ceph_fuse:config is None 2020-03-19T21:18:30.304 INFO:tasks.ceph_fuse:new config is {'client.1': None, 'client.0': None} 2020-03-19T21:18:30.304 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2020-03-19T21:18:30.305 INFO:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-03-19T21:18:30.417 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.416+0000 7f5648862700 1 Processor -- start 2020-03-19T21:18:30.417 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.416+0000 7f5648862700 1 -- start start 2020-03-19T21:18:30.423 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.416+0000 7f5648862700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f5644090ed0 0x7f564408e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T21:18:30.423 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.416+0000 7f5648862700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5644060a40 con 0x7f56440903e0 2020-03-19T21:18:30.423 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.416+0000 7f5648862700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5644060500 con 0x7f5644090ed0 2020-03-19T21:18:30.423 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.417+0000 7f563bfff700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f5644090ed0 0x7f564408e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:30.423 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.417+0000 7f563bfff700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f5644090ed0 0x7f564408e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53208/0 (socket says 172.21.15.170:53208) 2020-03-19T21:18:30.423 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.417+0000 7f56437fe700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f56440903e0 0x7f56440907c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:53972/0 (socket says 172.21.15.170:53972) 2020-03-19T21:18:30.423 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.417+0000 7f563bfff700 1 -- 172.21.15.170:0/3049257826 learned_addr learned my addr 172.21.15.170:0/3049257826 (peer_addr_for_me v2:172.21.15.170:0/0) 2020-03-19T21:18:30.424 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.417+0000 7f563bfff700 1 -- 172.21.15.170:0/3049257826 >> v1:172.21.15.170:6789/0 conn(0x7f56440903e0 legacy=0x7f56440907c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:30.424 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.417+0000 7f563bfff700 1 -- 172.21.15.170:0/3049257826 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5644060730 con 0x7f5644090ed0 2020-03-19T21:18:30.424 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.417+0000 7f5642ffd700 1 -- 172.21.15.170:0/3049257826 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1638428332 0 0) 0x7f5644060a40 con 0x7f56440903e0 2020-03-19T21:18:30.424 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.417+0000 7f563bfff700 1 --2- 172.21.15.170:0/3049257826 >> v2:172.21.15.170:3300/0 conn(0x7f5644090ed0 0x7f564408e720 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f562c002f30 tx=0x7f562c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:30.424 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.417+0000 7f563bfff700 1 --2- 172.21.15.170:0/3049257826 >> v2:172.21.15.170:3300/0 conn(0x7f5644090ed0 0x7f564408e720 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f562c002f30 tx=0x7f562c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:30.424 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.417+0000 7f563bfff700 1 --2- 172.21.15.170:0/3049257826 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5644090ed0 0x7f564408e720 secure :-1 s=READY pgs=231 cs=0 l=1 rx=0x7f562c002f30 tx=0x7f562c007a40).ready entity=mon.0 client_cookie=163acbe703567e3a server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:30.424 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.417+0000 7f563bfff700 1 --2- 172.21.15.170:0/3049257826 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5644090ed0 0x7f564408e720 secure :-1 s=THROTTLE_DONE pgs=231 cs=0 l=1 rx=0x7f562c002f30 tx=0x7f562c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:30.425 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.417+0000 7f5642ffd700 1 -- 172.21.15.170:0/3049257826 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f562c0114b0 con 0x7f5644090ed0 2020-03-19T21:18:30.425 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.417+0000 7f563bfff700 1 --2- 172.21.15.170:0/3049257826 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5644090ed0 0x7f564408e720 secure :-1 s=THROTTLE_DONE pgs=231 cs=0 l=1 rx=0x7f562c002f30 tx=0x7f562c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:30.425 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.417+0000 7f5642ffd700 1 -- 172.21.15.170:0/3049257826 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f562c011610 con 0x7f5644090ed0 2020-03-19T21:18:30.425 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.417+0000 7f563bfff700 1 --2- 172.21.15.170:0/3049257826 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5644090ed0 0x7f564408e720 secure :-1 s=THROTTLE_DONE pgs=231 cs=0 l=1 rx=0x7f562c002f30 tx=0x7f562c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:30.425 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.417+0000 7f5648862700 1 -- 172.21.15.170:0/3049257826 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5644090ed0 msgr2=0x7f564408e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:30.425 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.417+0000 7f5642ffd700 1 -- 172.21.15.170:0/3049257826 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f562c011930 con 0x7f5644090ed0 2020-03-19T21:18:30.425 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.417+0000 7f5648862700 1 --2- 172.21.15.170:0/3049257826 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5644090ed0 0x7f564408e720 secure :-1 s=READY pgs=231 cs=0 l=1 rx=0x7f562c002f30 tx=0x7f562c007a40).stop 2020-03-19T21:18:30.426 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.418+0000 7f5648862700 1 -- 172.21.15.170:0/3049257826 shutdown_connections 2020-03-19T21:18:30.426 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.418+0000 7f5648862700 1 --2- 172.21.15.170:0/3049257826 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5644090ed0 0x7f564408e720 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:30.426 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.418+0000 7f5648862700 1 -- 172.21.15.170:0/3049257826 >> 172.21.15.170:0/3049257826 conn(0x7f56440953a0 msgr2=0x7f56440914d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:30.426 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.418+0000 7f5648862700 1 -- 172.21.15.170:0/3049257826 shutdown_connections 2020-03-19T21:18:30.426 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.418+0000 7f5648862700 1 -- 172.21.15.170:0/3049257826 wait complete. 2020-03-19T21:18:30.426 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.418+0000 7f5648862700 1 Processor -- start 2020-03-19T21:18:30.426 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.418+0000 7f5648862700 1 -- start start 2020-03-19T21:18:30.427 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.419+0000 7f5648862700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f5644090ed0 0x7f5644081250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:30.427 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.419+0000 7f5648862700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5644067520 con 0x7f56440903e0 2020-03-19T21:18:30.427 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.419+0000 7f5648862700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56440875f0 con 0x7f5644090ed0 2020-03-19T21:18:30.427 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.419+0000 7f56437fe700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f56440903e0 0x7f5644080b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:53976/0 (socket says 172.21.15.170:53976) 2020-03-19T21:18:30.427 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.419+0000 7f56437fe700 1 -- 172.21.15.170:0/3825139357 learned_addr learned my addr 172.21.15.170:0/3825139357 (peer_addr_for_me v1:172.21.15.170:0/0) 2020-03-19T21:18:30.427 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.419+0000 7f563bfff700 1 --2- 172.21.15.170:0/3825139357 >> v2:172.21.15.170:3300/0 conn(0x7f5644090ed0 0x7f5644081250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:30.427 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.419+0000 7f56417fa700 1 -- 172.21.15.170:0/3825139357 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3116177277 0 0) 0x7f5644067520 con 0x7f56440903e0 2020-03-19T21:18:30.427 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.419+0000 7f56417fa700 1 -- 172.21.15.170:0/3825139357 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5628002fe0 con 0x7f56440903e0 2020-03-19T21:18:30.428 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.419+0000 7f563bfff700 1 -- 172.21.15.170:0/3825139357 >> v1:172.21.15.170:6789/0 conn(0x7f56440903e0 legacy=0x7f5644080b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:30.428 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.419+0000 7f563bfff700 1 -- 172.21.15.170:0/3825139357 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f564408c970 con 0x7f5644090ed0 2020-03-19T21:18:30.428 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.419+0000 7f563bfff700 1 --2- 172.21.15.170:0/3825139357 >> v2:172.21.15.170:3300/0 conn(0x7f5644090ed0 0x7f5644081250 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f562c002f30 tx=0x7f562c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:30.428 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.419+0000 7f563bfff700 1 --2- 172.21.15.170:0/3825139357 >> v2:172.21.15.170:3300/0 conn(0x7f5644090ed0 0x7f5644081250 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f562c002f30 tx=0x7f562c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:30.428 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.419+0000 7f563bfff700 1 --2- 172.21.15.170:0/3825139357 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5644090ed0 0x7f5644081250 secure :-1 s=READY pgs=233 cs=0 l=1 rx=0x7f562c002f30 tx=0x7f562c007a40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:30.428 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.419+0000 7f563bfff700 1 --2- 172.21.15.170:0/3825139357 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5644090ed0 0x7f5644081250 secure :-1 s=THROTTLE_DONE pgs=233 cs=0 l=1 rx=0x7f562c002f30 tx=0x7f562c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:30.428 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.419+0000 7f56417fa700 1 -- 172.21.15.170:0/3825139357 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f562c00f740 con 0x7f5644090ed0 2020-03-19T21:18:30.429 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.419+0000 7f563bfff700 1 --2- 172.21.15.170:0/3825139357 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5644090ed0 0x7f5644081250 secure :-1 s=THROTTLE_DONE pgs=233 cs=0 l=1 rx=0x7f562c002f30 tx=0x7f562c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:30.429 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.419+0000 7f563bfff700 1 --2- 172.21.15.170:0/3825139357 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5644090ed0 0x7f5644081250 secure :-1 s=THROTTLE_DONE pgs=233 cs=0 l=1 rx=0x7f562c002f30 tx=0x7f562c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:30.429 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.419+0000 7f56417fa700 1 -- 172.21.15.170:0/3825139357 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f562c00f8a0 con 0x7f5644090ed0 2020-03-19T21:18:30.429 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.419+0000 7f5648862700 1 -- 172.21.15.170:0/3825139357 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5644082130 con 0x7f5644090ed0 2020-03-19T21:18:30.429 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.419+0000 7f56417fa700 1 -- 172.21.15.170:0/3825139357 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f562c007280 con 0x7f5644090ed0 2020-03-19T21:18:30.429 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.419+0000 7f5648862700 1 -- 172.21.15.170:0/3825139357 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5644087110 con 0x7f5644090ed0 2020-03-19T21:18:30.429 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.420+0000 7f563bfff700 1 --2- 172.21.15.170:0/3825139357 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5644090ed0 0x7f5644081250 secure :-1 s=THROTTLE_DONE pgs=233 cs=0 l=1 rx=0x7f562c002f30 tx=0x7f562c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:30.430 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.420+0000 7f56417fa700 1 -- 172.21.15.170:0/3825139357 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 4) v1 ==== 29603+0+0 (secure 0 0 0) 0x7f562c0074c0 con 0x7f5644090ed0 2020-03-19T21:18:30.430 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.420+0000 7f563bfff700 1 --2- 172.21.15.170:0/3825139357 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5644090ed0 0x7f5644081250 secure :-1 s=THROTTLE_DONE pgs=233 cs=0 l=1 rx=0x7f562c002f30 tx=0x7f562c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:30.430 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.420+0000 7f56417fa700 1 --2- 172.21.15.170:0/3825139357 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f5628028720 0x7f562802abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:30.430 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.420+0000 7f56417fa700 1 -- 172.21.15.170:0/3825139357 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 3786+0+0 (secure 0 0 0) 0x7f562c0348a0 con 0x7f5644090ed0 2020-03-19T21:18:30.430 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.420+0000 7f5648862700 1 -- 172.21.15.170:0/3825139357 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5644056db0 con 0x7f5644090ed0 2020-03-19T21:18:30.430 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.420+0000 7f56437fe700 1 --2- 172.21.15.170:0/3825139357 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f5628028720 0x7f562802abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:30.430 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.421+0000 7f56437fe700 1 --2- 172.21.15.170:0/3825139357 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f5628028720 0x7f562802abe0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f5634000ff0 tx=0x7f5634001780).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:30.431 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.421+0000 7f56437fe700 1 --2- 172.21.15.170:0/3825139357 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f5628028720 0x7f562802abe0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f5634000ff0 tx=0x7f5634001780).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:30.431 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.421+0000 7f56437fe700 1 --2- 172.21.15.170:0/3825139357 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f5628028720 0x7f562802abe0 secure :-1 s=READY pgs=58 cs=0 l=1 rx=0x7f5634000ff0 tx=0x7f5634001780).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:30.431 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.424+0000 7f563bfff700 1 --2- 172.21.15.170:0/3825139357 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5644090ed0 0x7f5644081250 secure :-1 s=THROTTLE_DONE pgs=233 cs=0 l=1 rx=0x7f562c002f30 tx=0x7f562c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:30.431 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.424+0000 7f56417fa700 1 -- 172.21.15.170:0/3825139357 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+126772 (secure 0 0 0) 0x7f562c009820 con 0x7f5644090ed0 2020-03-19T21:18:30.567 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.566+0000 7f5648862700 1 -- 172.21.15.170:0/3825139357 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f564406aaf0 con 0x7f5644090ed0 2020-03-19T21:18:30.567 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.566+0000 7f563bfff700 1 --2- 172.21.15.170:0/3825139357 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5644090ed0 0x7f5644081250 secure :-1 s=THROTTLE_DONE pgs=233 cs=0 l=1 rx=0x7f562c002f30 tx=0x7f562c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:30.568 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.566+0000 7f56417fa700 1 -- 172.21.15.170:0/3825139357 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+10376 (secure 0 0 0) 0x7f562c009820 con 0x7f5644090ed0 2020-03-19T21:18:30.568 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:18:30.568 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":24,"fsid":"12d0a33b-6b40-4398-90b4-6b5df1da388e","created":"2020-03-19T21:17:59.135747+0000","modified":"2020-03-19T21:18:30.247068+0000","last_up_change":"2020-03-19T21:18:05.158515+0000","last_in_change":"2020-03-19T21:18:05.158515+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":3,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2020-03-19T21:18:08.391604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2020-03-19T21:18:11.286536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2020-03-19T21:18:12.597110+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"20","last_force_op_resend_prenautilus":"20","last_force_op_resend_preluminous":"20","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":18,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[4],"tier_of":-1,"read_tier":4,"write_tier":4,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":4,"pool_name":"data_cache","create_time":"2020-03-19T21:18:22.941000+0000","flags":9,"flags_names":"hashpspool,incomplete_clones","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":4,"pg_placement_num":4,"pg_placement_num_target":4,"pg_num_target":4,"pg_num_pending":4,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"20","last_force_op_resend_prenautilus":"20","last_force_op_resend_preluminous":"20","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":18,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":3,"read_tier":-1,"write_tier":-1,"cache_mode":"writeback","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"bloom","false_positive_probability":0.050000000000000003,"target_size":0,"seed":0},"hit_set_period":3600,"hit_set_count":8,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[1000000,1000000,1000000,1000000,1000000,1000000,1000000,1000000],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{}}],"osds":[{"osd":0,"uuid":"32ca3517-1077-4102-9aa4-edfcebe8e3d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":20680},{"type":"v1","addr":"172.21.15.170:6801","nonce":20680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":20680},{"type":"v1","addr":"172.21.15.170:6803","nonce":20680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":20680},{"type":"v1","addr":"172.21.15.170:6807","nonce":20680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":20680},{"type":"v1","addr":"172.21.15.170:6805","nonce":20680}]},"public_addr":"172.21.15.170:6801/20680","cluster_addr":"172.21.15.170:6803/20680","heartbeat_back_addr":"172.21.15.170:6807/20680","heartbeat_front_addr":"172.21.15.170:6805/20680","state":["exists","up"]},{"osd":1,"uuid":"470df64b-60bc-4f5b-9b53-64495cd7c5e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":20681},{"type":"v1","addr":"172.21.15.170:6809","nonce":20681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":20681},{"type":"v1","addr":"172.21.15.170:6811","nonce":20681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":20681},{"type":"v1","addr":"172.21.15.170:6815","nonce":20681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":20681},{"type":"v1","addr":"172.21.15.170:6813","nonce":20681}]},"public_addr":"172.21.15.170:6809/20681","cluster_addr":"172.21.15.170:6811/20681","heartbeat_back_addr":"172.21.15.170:6815/20681","heartbeat_front_addr":"172.21.15.170:6813/20681","state":["exists","up"]},{"osd":2,"uuid":"ccfb9d82-b759-4ff4-8ddf-4f1d741bac5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":20682},{"type":"v1","addr":"172.21.15.170:6817","nonce":20682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":20682},{"type":"v1","addr":"172.21.15.170:6819","nonce":20682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":20682},{"type":"v1","addr":"172.21.15.170:6823","nonce":20682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":20682},{"type":"v1","addr":"172.21.15.170:6821","nonce":20682}]},"public_addr":"172.21.15.170:6817/20682","cluster_addr":"172.21.15.170:6819/20682","heartbeat_back_addr":"172.21.15.170:6823/20682","heartbeat_front_addr":"172.21.15.170:6821/20682","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2020-03-19T21:18:03.177293+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2020-03-19T21:18:03.263179+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2020-03-19T21:18:03.183223+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-03-19T21:18:30.569 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.568+0000 7f5648862700 1 -- 172.21.15.170:0/3825139357 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f5628028720 msgr2=0x7f562802abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:30.569 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.568+0000 7f5648862700 1 --2- 172.21.15.170:0/3825139357 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f5628028720 0x7f562802abe0 secure :-1 s=READY pgs=58 cs=0 l=1 rx=0x7f5634000ff0 tx=0x7f5634001780).stop 2020-03-19T21:18:30.569 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.568+0000 7f5648862700 1 -- 172.21.15.170:0/3825139357 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5644090ed0 msgr2=0x7f5644081250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:30.570 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.568+0000 7f5648862700 1 --2- 172.21.15.170:0/3825139357 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5644090ed0 0x7f5644081250 secure :-1 s=READY pgs=233 cs=0 l=1 rx=0x7f562c002f30 tx=0x7f562c007a40).stop 2020-03-19T21:18:30.570 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.568+0000 7f5648862700 1 -- 172.21.15.170:0/3825139357 shutdown_connections 2020-03-19T21:18:30.570 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.568+0000 7f5648862700 1 --2- 172.21.15.170:0/3825139357 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f5628028720 0x7f562802abe0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:30.570 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.568+0000 7f5648862700 1 --2- 172.21.15.170:0/3825139357 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5644090ed0 0x7f5644081250 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:30.570 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.568+0000 7f5648862700 1 -- 172.21.15.170:0/3825139357 >> 172.21.15.170:0/3825139357 conn(0x7f56440953a0 msgr2=0x7f5644062bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:30.570 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.568+0000 7f5648862700 1 -- 172.21.15.170:0/3825139357 shutdown_connections 2020-03-19T21:18:30.570 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.568+0000 7f5648862700 1 -- 172.21.15.170:0/3825139357 wait complete. 2020-03-19T21:18:30.579 INFO:tasks.cephfs.filesystem.ceph_manager:[{u'cache_target_full_ratio_micro': 800000, u'fast_read': False, u'stripe_width': 0, u'flags_names': u'hashpspool', u'tier_of': -1, u'hit_set_grade_decay_rate': 0, u'pg_placement_num': 8, u'use_gmt_hitset': True, u'cache_target_dirty_ratio_micro': 400000, u'create_time': u'2020-03-19T21:18:08.391604+0000', u'snap_epoch': 0, u'erasure_code_profile': u'', u'pg_autoscale_mode': u'off', u'snap_seq': 0, u'expected_num_objects': 0, u'size': 2, u'pg_num_pending': 8, u'auid': 0, u'cache_min_flush_age': 0, u'hit_set_period': 0, u'min_read_recency_for_promote': 0, u'target_max_objects': 0, u'pg_placement_num_target': 8, u'pg_num': 8, u'type': 1, u'grade_table': [], u'pool_name': u'rbd', u'cache_min_evict_age': 0, u'snap_mode': u'selfmanaged', u'pg_num_target': 8, u'cache_mode': u'none', u'min_size': 1, u'cache_target_dirty_high_ratio_micro': 600000, u'object_hash': 2, u'last_pg_merge_meta': {u'ready_epoch': 0, u'source_version': u"0'0", u'source_pgid': u'0.0', u'last_epoch_clean': 0, u'target_version': u"0'0", u'last_epoch_started': 0}, u'write_tier': -1, u'last_force_op_resend_preluminous': u'0', u'pool': 1, u'removed_snaps': u'[]', u'crush_rule': 0, u'tiers': [], u'hit_set_params': {u'type': u'none'}, u'last_force_op_resend': u'0', u'pool_snaps': [], u'quota_max_objects': 0, u'last_force_op_resend_prenautilus': u'0', u'application_metadata': {u'rbd': {}}, u'options': {}, u'hit_set_count': 0, u'flags': 1, u'target_max_bytes': 0, u'quota_max_bytes': 0, u'hit_set_search_last_n': 0, u'last_change': u'12', u'min_write_recency_for_promote': 0, u'read_tier': -1}, {u'cache_target_full_ratio_micro': 800000, u'fast_read': False, u'stripe_width': 0, u'flags_names': u'hashpspool', u'tier_of': -1, u'hit_set_grade_decay_rate': 0, u'pg_placement_num': 8, u'use_gmt_hitset': True, u'cache_target_dirty_ratio_micro': 400000, u'create_time': u'2020-03-19T21:18:11.286536+0000', u'snap_epoch': 0, u'erasure_code_profile': u'', u'pg_autoscale_mode': u'off', u'snap_seq': 0, u'expected_num_objects': 0, u'size': 2, u'pg_num_pending': 8, u'auid': 0, u'cache_min_flush_age': 0, u'hit_set_period': 0, u'min_read_recency_for_promote': 0, u'target_max_objects': 0, u'pg_placement_num_target': 8, u'pg_num': 8, u'type': 1, u'grade_table': [], u'pool_name': u'cephfs_metadata', u'cache_min_evict_age': 0, u'snap_mode': u'selfmanaged', u'pg_num_target': 8, u'cache_mode': u'none', u'min_size': 1, u'cache_target_dirty_high_ratio_micro': 600000, u'object_hash': 2, u'last_pg_merge_meta': {u'ready_epoch': 0, u'source_version': u"0'0", u'source_pgid': u'0.0', u'last_epoch_clean': 0, u'target_version': u"0'0", u'last_epoch_started': 0}, u'write_tier': -1, u'last_force_op_resend_preluminous': u'0', u'pool': 2, u'removed_snaps': u'[]', u'crush_rule': 0, u'tiers': [], u'hit_set_params': {u'type': u'none'}, u'last_force_op_resend': u'0', u'pool_snaps': [], u'quota_max_objects': 0, u'last_force_op_resend_prenautilus': u'0', u'application_metadata': {u'cephfs': {u'metadata': u'cephfs'}}, u'options': {u'pg_autoscale_bias': 4, u'recovery_priority': 5, u'pg_num_min': 16}, u'hit_set_count': 0, u'flags': 1, u'target_max_bytes': 0, u'quota_max_bytes': 0, u'hit_set_search_last_n': 0, u'last_change': u'15', u'min_write_recency_for_promote': 0, u'read_tier': -1}, {u'cache_target_full_ratio_micro': 800000, u'fast_read': False, u'stripe_width': 0, u'flags_names': u'hashpspool', u'tier_of': -1, u'hit_set_grade_decay_rate': 0, u'pg_placement_num': 8, u'use_gmt_hitset': True, u'cache_target_dirty_ratio_micro': 400000, u'create_time': u'2020-03-19T21:18:12.597110+0000', u'snap_epoch': 18, u'erasure_code_profile': u'', u'pg_autoscale_mode': u'off', u'snap_seq': 0, u'expected_num_objects': 0, u'size': 2, u'pg_num_pending': 8, u'auid': 0, u'cache_min_flush_age': 0, u'hit_set_period': 0, u'min_read_recency_for_promote': 0, u'target_max_objects': 0, u'pg_placement_num_target': 8, u'pg_num': 8, u'type': 1, u'grade_table': [], u'pool_name': u'cephfs_data', u'cache_min_evict_age': 0, u'snap_mode': u'selfmanaged', u'pg_num_target': 8, u'cache_mode': u'none', u'min_size': 1, u'cache_target_dirty_high_ratio_micro': 600000, u'object_hash': 2, u'last_pg_merge_meta': {u'ready_epoch': 0, u'source_version': u"0'0", u'source_pgid': u'0.0', u'last_epoch_clean': 0, u'target_version': u"0'0", u'last_epoch_started': 0}, u'write_tier': 4, u'last_force_op_resend_preluminous': u'20', u'pool': 3, u'removed_snaps': u'[]', u'crush_rule': 0, u'tiers': [4], u'hit_set_params': {u'type': u'none'}, u'last_force_op_resend': u'20', u'pool_snaps': [], u'quota_max_objects': 0, u'last_force_op_resend_prenautilus': u'20', u'application_metadata': {u'cephfs': {u'data': u'cephfs'}}, u'options': {}, u'hit_set_count': 0, u'flags': 1, u'target_max_bytes': 0, u'quota_max_bytes': 0, u'hit_set_search_last_n': 0, u'last_change': u'20', u'min_write_recency_for_promote': 0, u'read_tier': 4}, {u'cache_target_full_ratio_micro': 800000, u'fast_read': False, u'stripe_width': 0, u'flags_names': u'hashpspool,incomplete_clones', u'tier_of': 3, u'hit_set_grade_decay_rate': 0, u'pg_placement_num': 4, u'use_gmt_hitset': True, u'cache_target_dirty_ratio_micro': 400000, u'create_time': u'2020-03-19T21:18:22.941000+0000', u'snap_epoch': 18, u'erasure_code_profile': u'', u'pg_autoscale_mode': u'off', u'snap_seq': 0, u'expected_num_objects': 0, u'size': 2, u'pg_num_pending': 4, u'auid': 0, u'cache_min_flush_age': 0, u'hit_set_period': 3600, u'min_read_recency_for_promote': 0, u'target_max_objects': 0, u'pg_placement_num_target': 4, u'pg_num': 4, u'type': 1, u'grade_table': [1000000, 1000000, 1000000, 1000000, 1000000, 1000000, 1000000, 1000000], u'pool_name': u'data_cache', u'cache_min_evict_age': 0, u'snap_mode': u'selfmanaged', u'pg_num_target': 4, u'cache_mode': u'writeback', u'min_size': 1, u'cache_target_dirty_high_ratio_micro': 600000, u'object_hash': 2, u'last_pg_merge_meta': {u'ready_epoch': 0, u'source_version': u"0'0", u'source_pgid': u'0.0', u'last_epoch_clean': 0, u'target_version': u"0'0", u'last_epoch_started': 0}, u'write_tier': -1, u'last_force_op_resend_preluminous': u'20', u'pool': 4, u'removed_snaps': u'[]', u'crush_rule': 0, u'tiers': [], u'hit_set_params': {u'target_size': 0, u'seed': 0, u'type': u'bloom', u'false_positive_probability': 0.05}, u'last_force_op_resend': u'20', u'pool_snaps': [], u'quota_max_objects': 0, u'last_force_op_resend_prenautilus': u'20', u'application_metadata': {}, u'options': {}, u'hit_set_count': 8, u'flags': 9, u'target_max_bytes': 0, u'quota_max_bytes': 0, u'hit_set_search_last_n': 0, u'last_change': u'24', u'min_write_recency_for_promote': 0, u'read_tier': -1}] 2020-03-19T21:18:30.580 INFO:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-03-19T21:18:30.693 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.686+0000 7fd817669700 1 Processor -- start 2020-03-19T21:18:30.694 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.686+0000 7fd817669700 1 -- start start 2020-03-19T21:18:30.694 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.687+0000 7fd817669700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fd810125060 0x7fd810127450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T21:18:30.694 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.687+0000 7fd817669700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd810060f10 con 0x7fd810065970 2020-03-19T21:18:30.694 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.687+0000 7fd817669700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd810061490 con 0x7fd810125060 2020-03-19T21:18:30.694 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.687+0000 7fd815405700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fd810065970 0x7fd810065d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:53982/0 (socket says 172.21.15.170:53982) 2020-03-19T21:18:30.694 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.687+0000 7fd815405700 1 -- 172.21.15.170:0/3017475775 learned_addr learned my addr 172.21.15.170:0/3017475775 (peer_addr_for_me v1:172.21.15.170:0/0) 2020-03-19T21:18:30.695 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.687+0000 7fd814c04700 1 --2- 172.21.15.170:0/3017475775 >> v2:172.21.15.170:3300/0 conn(0x7fd810125060 0x7fd810127450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:30.695 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.687+0000 7fd7fffff700 1 -- 172.21.15.170:0/3017475775 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3694589564 0 0) 0x7fd8000010d0 con 0x7fd810065970 2020-03-19T21:18:30.695 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.687+0000 7fd7fffff700 1 -- 172.21.15.170:0/3017475775 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd7f8003230 con 0x7fd810065970 2020-03-19T21:18:30.695 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.687+0000 7fd814c04700 1 -- 172.21.15.170:0/3017475775 >> v1:172.21.15.170:6789/0 conn(0x7fd810065970 legacy=0x7fd810065d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:30.695 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.687+0000 7fd814c04700 1 -- 172.21.15.170:0/3017475775 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd810061730 con 0x7fd810125060 2020-03-19T21:18:30.695 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.687+0000 7fd814c04700 1 --2- 172.21.15.170:0/3017475775 >> v2:172.21.15.170:3300/0 conn(0x7fd810125060 0x7fd810127450 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7fd80c002f30 tx=0x7fd80c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:30.695 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.687+0000 7fd814c04700 1 --2- 172.21.15.170:0/3017475775 >> v2:172.21.15.170:3300/0 conn(0x7fd810125060 0x7fd810127450 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7fd80c002f30 tx=0x7fd80c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:30.696 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.687+0000 7fd814c04700 1 --2- 172.21.15.170:0/3017475775 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd810125060 0x7fd810127450 secure :-1 s=READY pgs=235 cs=0 l=1 rx=0x7fd80c002f30 tx=0x7fd80c007a40).ready entity=mon.0 client_cookie=f31d7680da2c0fcd server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:30.696 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.687+0000 7fd814c04700 1 --2- 172.21.15.170:0/3017475775 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd810125060 0x7fd810127450 secure :-1 s=THROTTLE_DONE pgs=235 cs=0 l=1 rx=0x7fd80c002f30 tx=0x7fd80c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:30.696 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.687+0000 7fd7fffff700 1 -- 172.21.15.170:0/3017475775 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fd80c011340 con 0x7fd810125060 2020-03-19T21:18:30.696 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.687+0000 7fd814c04700 1 --2- 172.21.15.170:0/3017475775 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd810125060 0x7fd810127450 secure :-1 s=THROTTLE_DONE pgs=235 cs=0 l=1 rx=0x7fd80c002f30 tx=0x7fd80c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:30.696 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.687+0000 7fd814c04700 1 --2- 172.21.15.170:0/3017475775 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd810125060 0x7fd810127450 secure :-1 s=THROTTLE_DONE pgs=235 cs=0 l=1 rx=0x7fd80c002f30 tx=0x7fd80c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:30.696 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.687+0000 7fd7fffff700 1 -- 172.21.15.170:0/3017475775 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd80c0114a0 con 0x7fd810125060 2020-03-19T21:18:30.696 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.687+0000 7fd7fffff700 1 -- 172.21.15.170:0/3017475775 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fd80c0117c0 con 0x7fd810125060 2020-03-19T21:18:30.696 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.687+0000 7fd817669700 1 -- 172.21.15.170:0/3017475775 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd810125060 msgr2=0x7fd810127450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:30.697 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.687+0000 7fd817669700 1 --2- 172.21.15.170:0/3017475775 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd810125060 0x7fd810127450 secure :-1 s=READY pgs=235 cs=0 l=1 rx=0x7fd80c002f30 tx=0x7fd80c007a40).stop 2020-03-19T21:18:30.697 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.688+0000 7fd817669700 1 -- 172.21.15.170:0/3017475775 shutdown_connections 2020-03-19T21:18:30.697 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.688+0000 7fd817669700 1 --2- 172.21.15.170:0/3017475775 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd810125060 0x7fd810127450 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:30.697 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.688+0000 7fd817669700 1 -- 172.21.15.170:0/3017475775 >> 172.21.15.170:0/3017475775 conn(0x7fd810091390 msgr2=0x7fd810091770 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:30.697 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.688+0000 7fd817669700 1 -- 172.21.15.170:0/3017475775 shutdown_connections 2020-03-19T21:18:30.697 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.688+0000 7fd817669700 1 -- 172.21.15.170:0/3017475775 wait complete. 2020-03-19T21:18:30.697 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.689+0000 7fd817669700 1 Processor -- start 2020-03-19T21:18:30.698 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.689+0000 7fd817669700 1 -- start start 2020-03-19T21:18:30.698 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.689+0000 7fd817669700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fd810065970 0x7fd81012e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:30.698 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.689+0000 7fd817669700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd81010eff0 con 0x7fd810125060 2020-03-19T21:18:30.698 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.689+0000 7fd817669700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd810124bc0 con 0x7fd810065970 2020-03-19T21:18:30.698 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.689+0000 7fd815405700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fd810065970 0x7fd81012e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:30.698 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.689+0000 7fd814c04700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fd810125060 0x7fd81010deb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:53986/0 (socket says 172.21.15.170:53986) 2020-03-19T21:18:30.698 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.689+0000 7fd815405700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fd810065970 0x7fd81012e9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53224/0 (socket says 172.21.15.170:53224) 2020-03-19T21:18:30.699 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.689+0000 7fd814c04700 1 -- 172.21.15.170:0/1017532225 learned_addr learned my addr 172.21.15.170:0/1017532225 (peer_addr_for_me v1:172.21.15.170:0/0) 2020-03-19T21:18:30.699 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.689+0000 7fd815405700 1 -- 172.21.15.170:0/1017532225 >> v1:172.21.15.170:6789/0 conn(0x7fd810125060 legacy=0x7fd81010deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:30.699 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.689+0000 7fd815405700 1 -- 172.21.15.170:0/1017532225 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd810125750 con 0x7fd810065970 2020-03-19T21:18:30.699 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.689+0000 7fd7fe7fc700 1 -- 172.21.15.170:0/1017532225 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 141678365 0 0) 0x7fd81010eff0 con 0x7fd810125060 2020-03-19T21:18:30.699 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.690+0000 7fd815405700 1 --2- 172.21.15.170:0/1017532225 >> v2:172.21.15.170:3300/0 conn(0x7fd810065970 0x7fd81012e9e0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7fd800001260 tx=0x7fd8000108c0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:30.699 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.690+0000 7fd815405700 1 --2- 172.21.15.170:0/1017532225 >> v2:172.21.15.170:3300/0 conn(0x7fd810065970 0x7fd81012e9e0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7fd800001260 tx=0x7fd8000108c0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:30.699 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.690+0000 7fd815405700 1 --2- 172.21.15.170:0/1017532225 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd810065970 0x7fd81012e9e0 secure :-1 s=READY pgs=237 cs=0 l=1 rx=0x7fd800001260 tx=0x7fd8000108c0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:30.700 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.690+0000 7fd815405700 1 --2- 172.21.15.170:0/1017532225 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd810065970 0x7fd81012e9e0 secure :-1 s=THROTTLE_DONE pgs=237 cs=0 l=1 rx=0x7fd800001260 tx=0x7fd8000108c0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:30.700 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.690+0000 7fd7fe7fc700 1 -- 172.21.15.170:0/1017532225 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fd800005b40 con 0x7fd810065970 2020-03-19T21:18:30.700 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.690+0000 7fd815405700 1 --2- 172.21.15.170:0/1017532225 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd810065970 0x7fd81012e9e0 secure :-1 s=THROTTLE_DONE pgs=237 cs=0 l=1 rx=0x7fd800001260 tx=0x7fd8000108c0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:30.700 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.690+0000 7fd815405700 1 --2- 172.21.15.170:0/1017532225 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd810065970 0x7fd81012e9e0 secure :-1 s=THROTTLE_DONE pgs=237 cs=0 l=1 rx=0x7fd800001260 tx=0x7fd8000108c0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:30.700 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.690+0000 7fd7fe7fc700 1 -- 172.21.15.170:0/1017532225 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd800005ca0 con 0x7fd810065970 2020-03-19T21:18:30.700 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.690+0000 7fd817669700 1 -- 172.21.15.170:0/1017532225 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8101254a0 con 0x7fd810065970 2020-03-19T21:18:30.700 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.690+0000 7fd7fe7fc700 1 -- 172.21.15.170:0/1017532225 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fd8000060c0 con 0x7fd810065970 2020-03-19T21:18:30.700 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.690+0000 7fd817669700 1 -- 172.21.15.170:0/1017532225 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd810124e10 con 0x7fd810065970 2020-03-19T21:18:30.701 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.690+0000 7fd815405700 1 --2- 172.21.15.170:0/1017532225 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd810065970 0x7fd81012e9e0 secure :-1 s=THROTTLE_DONE pgs=237 cs=0 l=1 rx=0x7fd800001260 tx=0x7fd8000108c0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:30.701 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.691+0000 7fd7fe7fc700 1 -- 172.21.15.170:0/1017532225 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 4) v1 ==== 29603+0+0 (secure 0 0 0) 0x7fd80000a320 con 0x7fd810065970 2020-03-19T21:18:30.701 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.691+0000 7fd815405700 1 --2- 172.21.15.170:0/1017532225 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd810065970 0x7fd81012e9e0 secure :-1 s=THROTTLE_DONE pgs=237 cs=0 l=1 rx=0x7fd800001260 tx=0x7fd8000108c0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:30.701 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.691+0000 7fd7fe7fc700 1 --2- 172.21.15.170:0/1017532225 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7fd7f0028230 0x7fd7f002a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:30.701 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.691+0000 7fd7fe7fc700 1 -- 172.21.15.170:0/1017532225 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 3786+0+0 (secure 0 0 0) 0x7fd800033ee0 con 0x7fd810065970 2020-03-19T21:18:30.701 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.691+0000 7fd814c04700 1 --2- 172.21.15.170:0/1017532225 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7fd7f0028230 0x7fd7f002a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:30.701 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.691+0000 7fd814c04700 1 --2- 172.21.15.170:0/1017532225 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7fd7f0028230 0x7fd7f002a6f0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7fd80c010110 tx=0x7fd80c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:30.702 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.691+0000 7fd814c04700 1 --2- 172.21.15.170:0/1017532225 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7fd7f0028230 0x7fd7f002a6f0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7fd80c010110 tx=0x7fd80c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:30.702 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.691+0000 7fd817669700 1 -- 172.21.15.170:0/1017532225 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd810060f10 con 0x7fd810065970 2020-03-19T21:18:30.702 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.691+0000 7fd814c04700 1 --2- 172.21.15.170:0/1017532225 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7fd7f0028230 0x7fd7f002a6f0 secure :-1 s=READY pgs=59 cs=0 l=1 rx=0x7fd80c010110 tx=0x7fd80c007a40).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:30.702 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.694+0000 7fd815405700 1 --2- 172.21.15.170:0/1017532225 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd810065970 0x7fd81012e9e0 secure :-1 s=THROTTLE_DONE pgs=237 cs=0 l=1 rx=0x7fd800001260 tx=0x7fd8000108c0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:30.702 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.695+0000 7fd7fe7fc700 1 -- 172.21.15.170:0/1017532225 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+126772 (secure 0 0 0) 0x7fd810060f10 con 0x7fd810065970 2020-03-19T21:18:30.836 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.835+0000 7fd817669700 1 -- 172.21.15.170:0/1017532225 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fd810060f10 con 0x7fd810065970 2020-03-19T21:18:30.836 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.835+0000 7fd815405700 1 --2- 172.21.15.170:0/1017532225 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd810065970 0x7fd81012e9e0 secure :-1 s=THROTTLE_DONE pgs=237 cs=0 l=1 rx=0x7fd800001260 tx=0x7fd8000108c0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:30.837 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.835+0000 7fd7fe7fc700 1 -- 172.21.15.170:0/1017532225 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v24) v1 ==== 92+0+10 (secure 0 0 0) 0x7fd810060f10 con 0x7fd810065970 2020-03-19T21:18:30.837 INFO:teuthology.orchestra.run.smithi170.stdout:pg_num: 8 2020-03-19T21:18:30.838 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.836+0000 7fd817669700 1 -- 172.21.15.170:0/1017532225 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7fd7f0028230 msgr2=0x7fd7f002a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:30.838 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.836+0000 7fd817669700 1 --2- 172.21.15.170:0/1017532225 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7fd7f0028230 0x7fd7f002a6f0 secure :-1 s=READY pgs=59 cs=0 l=1 rx=0x7fd80c010110 tx=0x7fd80c007a40).stop 2020-03-19T21:18:30.838 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.836+0000 7fd817669700 1 -- 172.21.15.170:0/1017532225 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd810065970 msgr2=0x7fd81012e9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:30.838 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.836+0000 7fd817669700 1 --2- 172.21.15.170:0/1017532225 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd810065970 0x7fd81012e9e0 secure :-1 s=READY pgs=237 cs=0 l=1 rx=0x7fd800001260 tx=0x7fd8000108c0).stop 2020-03-19T21:18:30.839 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.837+0000 7fd817669700 1 -- 172.21.15.170:0/1017532225 shutdown_connections 2020-03-19T21:18:30.839 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.837+0000 7fd817669700 1 --2- 172.21.15.170:0/1017532225 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7fd7f0028230 0x7fd7f002a6f0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:30.839 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.837+0000 7fd817669700 1 --2- 172.21.15.170:0/1017532225 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd810065970 0x7fd81012e9e0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:30.839 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.837+0000 7fd817669700 1 -- 172.21.15.170:0/1017532225 >> 172.21.15.170:0/1017532225 conn(0x7fd810091390 msgr2=0x7fd810093180 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:30.839 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.837+0000 7fd817669700 1 -- 172.21.15.170:0/1017532225 shutdown_connections 2020-03-19T21:18:30.839 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.837+0000 7fd817669700 1 -- 172.21.15.170:0/1017532225 wait complete. 2020-03-19T21:18:30.848 INFO:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2020-03-19T21:18:30.956 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.954+0000 7f444591a700 1 Processor -- start 2020-03-19T21:18:30.956 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.954+0000 7f444591a700 1 -- start start 2020-03-19T21:18:30.961 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.955+0000 7f444591a700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f44401224c0 0x7f4440122ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T21:18:30.961 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.955+0000 7f444591a700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44401228a0 con 0x7f4440122f40 2020-03-19T21:18:30.962 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.955+0000 7f444591a700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4440092200 con 0x7f44401224c0 2020-03-19T21:18:30.962 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.955+0000 7f443effd700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f44401224c0 0x7f4440122ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:30.962 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.955+0000 7f443e7fc700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f4440122f40 0x7f4440060110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:53992/0 (socket says 172.21.15.170:53992) 2020-03-19T21:18:30.962 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.955+0000 7f443effd700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f44401224c0 0x7f4440122ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53228/0 (socket says 172.21.15.170:53228) 2020-03-19T21:18:30.962 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.955+0000 7f443e7fc700 1 -- 172.21.15.170:0/1917545291 learned_addr learned my addr 172.21.15.170:0/1917545291 (peer_addr_for_me v1:172.21.15.170:0/0) 2020-03-19T21:18:30.962 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.956+0000 7f443effd700 1 -- 172.21.15.170:0/1917545291 >> v1:172.21.15.170:6789/0 conn(0x7f4440122f40 legacy=0x7f4440060110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:30.962 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.956+0000 7f443dffb700 1 -- 172.21.15.170:0/1917545291 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2485085333 0 0) 0x7f44300010d0 con 0x7f4440122f40 2020-03-19T21:18:30.963 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.956+0000 7f443effd700 1 -- 172.21.15.170:0/1917545291 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4440090b60 con 0x7f44401224c0 2020-03-19T21:18:30.963 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.956+0000 7f443effd700 1 --2- 172.21.15.170:0/1917545291 >> v2:172.21.15.170:3300/0 conn(0x7f44401224c0 0x7f4440122ae0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f4434002f30 tx=0x7f4434007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:30.963 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.956+0000 7f443effd700 1 --2- 172.21.15.170:0/1917545291 >> v2:172.21.15.170:3300/0 conn(0x7f44401224c0 0x7f4440122ae0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f4434002f30 tx=0x7f4434007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:30.963 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.956+0000 7f443effd700 1 --2- 172.21.15.170:0/1917545291 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f44401224c0 0x7f4440122ae0 secure :-1 s=READY pgs=239 cs=0 l=1 rx=0x7f4434002f30 tx=0x7f4434007a40).ready entity=mon.0 client_cookie=b6762c12b6b93ed8 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:30.963 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.956+0000 7f443effd700 1 --2- 172.21.15.170:0/1917545291 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f44401224c0 0x7f4440122ae0 secure :-1 s=THROTTLE_DONE pgs=239 cs=0 l=1 rx=0x7f4434002f30 tx=0x7f4434007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:30.963 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.956+0000 7f443dffb700 1 -- 172.21.15.170:0/1917545291 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f4434008e80 con 0x7f44401224c0 2020-03-19T21:18:30.963 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.956+0000 7f443effd700 1 --2- 172.21.15.170:0/1917545291 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f44401224c0 0x7f4440122ae0 secure :-1 s=THROTTLE_DONE pgs=239 cs=0 l=1 rx=0x7f4434002f30 tx=0x7f4434007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:30.964 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.956+0000 7f443dffb700 1 -- 172.21.15.170:0/1917545291 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44340113f0 con 0x7f44401224c0 2020-03-19T21:18:30.964 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.956+0000 7f443effd700 1 --2- 172.21.15.170:0/1917545291 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f44401224c0 0x7f4440122ae0 secure :-1 s=THROTTLE_DONE pgs=239 cs=0 l=1 rx=0x7f4434002f30 tx=0x7f4434007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:30.964 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.956+0000 7f443dffb700 1 -- 172.21.15.170:0/1917545291 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f44340116d0 con 0x7f44401224c0 2020-03-19T21:18:30.964 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.956+0000 7f444591a700 1 -- 172.21.15.170:0/1917545291 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f44401224c0 msgr2=0x7f4440122ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:30.964 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.956+0000 7f444591a700 1 --2- 172.21.15.170:0/1917545291 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f44401224c0 0x7f4440122ae0 secure :-1 s=READY pgs=239 cs=0 l=1 rx=0x7f4434002f30 tx=0x7f4434007a40).stop 2020-03-19T21:18:30.964 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.956+0000 7f444591a700 1 -- 172.21.15.170:0/1917545291 shutdown_connections 2020-03-19T21:18:30.964 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.956+0000 7f444591a700 1 --2- 172.21.15.170:0/1917545291 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f44401224c0 0x7f4440122ae0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:30.965 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.956+0000 7f444591a700 1 -- 172.21.15.170:0/1917545291 >> 172.21.15.170:0/1917545291 conn(0x7f4440091390 msgr2=0x7f4440091770 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:30.965 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.956+0000 7f444591a700 1 -- 172.21.15.170:0/1917545291 shutdown_connections 2020-03-19T21:18:30.965 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.956+0000 7f444591a700 1 -- 172.21.15.170:0/1917545291 wait complete. 2020-03-19T21:18:30.965 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.957+0000 7f444591a700 1 Processor -- start 2020-03-19T21:18:30.965 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.957+0000 7f444591a700 1 -- start start 2020-03-19T21:18:30.965 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.957+0000 7f444591a700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f44401224c0 0x7f4440061c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:30.965 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.958+0000 7f444591a700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4440063cc0 con 0x7f4440122f40 2020-03-19T21:18:30.966 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.958+0000 7f444591a700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4440065460 con 0x7f44401224c0 2020-03-19T21:18:30.966 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.958+0000 7f443effd700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f44401224c0 0x7f4440061c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:30.966 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.958+0000 7f443effd700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f44401224c0 0x7f4440061c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53232/0 (socket says 172.21.15.170:53232) 2020-03-19T21:18:30.966 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.958+0000 7f443e7fc700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f4440122f40 0x7f44400620b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:53998/0 (socket says 172.21.15.170:53998) 2020-03-19T21:18:30.966 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.958+0000 7f443effd700 1 -- 172.21.15.170:0/3450479149 learned_addr learned my addr 172.21.15.170:0/3450479149 (peer_addr_for_me v2:172.21.15.170:0/0) 2020-03-19T21:18:30.966 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.958+0000 7f443effd700 1 -- 172.21.15.170:0/3450479149 >> v1:172.21.15.170:6789/0 conn(0x7f4440122f40 legacy=0x7f44400620b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:30.966 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.958+0000 7f442ffff700 1 -- 172.21.15.170:0/3450479149 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1158752608 0 0) 0x7f4440063cc0 con 0x7f4440122f40 2020-03-19T21:18:30.967 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.958+0000 7f443effd700 1 -- 172.21.15.170:0/3450479149 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44400636c0 con 0x7f44401224c0 2020-03-19T21:18:30.967 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.958+0000 7f443effd700 1 --2- 172.21.15.170:0/3450479149 >> v2:172.21.15.170:3300/0 conn(0x7f44401224c0 0x7f4440061c50 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f4434002f30 tx=0x7f4434007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:30.967 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.958+0000 7f443effd700 1 --2- 172.21.15.170:0/3450479149 >> v2:172.21.15.170:3300/0 conn(0x7f44401224c0 0x7f4440061c50 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f4434002f30 tx=0x7f4434007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:30.967 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.958+0000 7f443effd700 1 --2- 172.21.15.170:0/3450479149 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f44401224c0 0x7f4440061c50 secure :-1 s=READY pgs=241 cs=0 l=1 rx=0x7f4434002f30 tx=0x7f4434007a40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:30.967 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.958+0000 7f443effd700 1 --2- 172.21.15.170:0/3450479149 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f44401224c0 0x7f4440061c50 secure :-1 s=THROTTLE_DONE pgs=241 cs=0 l=1 rx=0x7f4434002f30 tx=0x7f4434007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:30.967 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.958+0000 7f442ffff700 1 -- 172.21.15.170:0/3450479149 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f4434007150 con 0x7f44401224c0 2020-03-19T21:18:30.967 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.958+0000 7f443effd700 1 --2- 172.21.15.170:0/3450479149 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f44401224c0 0x7f4440061c50 secure :-1 s=THROTTLE_DONE pgs=241 cs=0 l=1 rx=0x7f4434002f30 tx=0x7f4434007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:30.968 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.958+0000 7f443effd700 1 --2- 172.21.15.170:0/3450479149 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f44401224c0 0x7f4440061c50 secure :-1 s=THROTTLE_DONE pgs=241 cs=0 l=1 rx=0x7f4434002f30 tx=0x7f4434007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:30.968 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.958+0000 7f442ffff700 1 -- 172.21.15.170:0/3450479149 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44340072b0 con 0x7f44401224c0 2020-03-19T21:18:30.968 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.958+0000 7f444591a700 1 -- 172.21.15.170:0/3450479149 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4440128120 con 0x7f44401224c0 2020-03-19T21:18:30.968 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.958+0000 7f442ffff700 1 -- 172.21.15.170:0/3450479149 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f4434007550 con 0x7f44401224c0 2020-03-19T21:18:30.968 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.958+0000 7f444591a700 1 -- 172.21.15.170:0/3450479149 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44400627c0 con 0x7f44401224c0 2020-03-19T21:18:30.968 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.959+0000 7f443effd700 1 --2- 172.21.15.170:0/3450479149 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f44401224c0 0x7f4440061c50 secure :-1 s=THROTTLE_DONE pgs=241 cs=0 l=1 rx=0x7f4434002f30 tx=0x7f4434007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:30.968 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.959+0000 7f442ffff700 1 -- 172.21.15.170:0/3450479149 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 4) v1 ==== 29603+0+0 (secure 0 0 0) 0x7f44340056c0 con 0x7f44401224c0 2020-03-19T21:18:30.969 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.959+0000 7f443effd700 1 --2- 172.21.15.170:0/3450479149 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f44401224c0 0x7f4440061c50 secure :-1 s=THROTTLE_DONE pgs=241 cs=0 l=1 rx=0x7f4434002f30 tx=0x7f4434007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:30.969 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.959+0000 7f442ffff700 1 --2- 172.21.15.170:0/3450479149 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f441c0282b0 0x7f441c02a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:30.969 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.959+0000 7f442ffff700 1 -- 172.21.15.170:0/3450479149 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 3786+0+0 (secure 0 0 0) 0x7f4434034720 con 0x7f44401224c0 2020-03-19T21:18:30.969 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.959+0000 7f443e7fc700 1 --2- 172.21.15.170:0/3450479149 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f441c0282b0 0x7f441c02a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:30.969 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.959+0000 7f444591a700 1 -- 172.21.15.170:0/3450479149 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4440090b60 con 0x7f44401224c0 2020-03-19T21:18:30.969 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.959+0000 7f443e7fc700 1 --2- 172.21.15.170:0/3450479149 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f441c0282b0 0x7f441c02a770 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f4430000ee0 tx=0x7f4430001700).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:30.969 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.959+0000 7f443e7fc700 1 --2- 172.21.15.170:0/3450479149 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f441c0282b0 0x7f441c02a770 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f4430000ee0 tx=0x7f4430001700).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:30.970 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.959+0000 7f443e7fc700 1 --2- 172.21.15.170:0/3450479149 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f441c0282b0 0x7f441c02a770 secure :-1 s=READY pgs=60 cs=0 l=1 rx=0x7f4430000ee0 tx=0x7f4430001700).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:30.970 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.963+0000 7f443effd700 1 --2- 172.21.15.170:0/3450479149 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f44401224c0 0x7f4440061c50 secure :-1 s=THROTTLE_DONE pgs=241 cs=0 l=1 rx=0x7f4434002f30 tx=0x7f4434007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:30.970 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:30.963+0000 7f442ffff700 1 -- 172.21.15.170:0/3450479149 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+126772 (secure 0 0 0) 0x7f4440090b60 con 0x7f44401224c0 2020-03-19T21:18:31.101 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.100+0000 7f444591a700 1 -- 172.21.15.170:0/3450479149 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4440090b60 con 0x7f44401224c0 2020-03-19T21:18:31.101 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.100+0000 7f443effd700 1 --2- 172.21.15.170:0/3450479149 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f44401224c0 0x7f4440061c50 secure :-1 s=THROTTLE_DONE pgs=241 cs=0 l=1 rx=0x7f4434002f30 tx=0x7f4434007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:31.102 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.100+0000 7f442ffff700 1 -- 172.21.15.170:0/3450479149 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+10 (secure 0 0 0) 0x7f4434014020 con 0x7f44401224c0 2020-03-19T21:18:31.102 INFO:teuthology.orchestra.run.smithi170.stdout:pg_num: 8 2020-03-19T21:18:31.103 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.101+0000 7f444591a700 1 -- 172.21.15.170:0/3450479149 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f441c0282b0 msgr2=0x7f441c02a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:31.103 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.101+0000 7f444591a700 1 --2- 172.21.15.170:0/3450479149 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f441c0282b0 0x7f441c02a770 secure :-1 s=READY pgs=60 cs=0 l=1 rx=0x7f4430000ee0 tx=0x7f4430001700).stop 2020-03-19T21:18:31.103 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.101+0000 7f444591a700 1 -- 172.21.15.170:0/3450479149 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f44401224c0 msgr2=0x7f4440061c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:31.103 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.101+0000 7f444591a700 1 --2- 172.21.15.170:0/3450479149 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f44401224c0 0x7f4440061c50 secure :-1 s=READY pgs=241 cs=0 l=1 rx=0x7f4434002f30 tx=0x7f4434007a40).stop 2020-03-19T21:18:31.103 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.101+0000 7f444591a700 1 -- 172.21.15.170:0/3450479149 shutdown_connections 2020-03-19T21:18:31.103 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.101+0000 7f444591a700 1 --2- 172.21.15.170:0/3450479149 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f441c0282b0 0x7f441c02a770 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:31.104 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.101+0000 7f444591a700 1 --2- 172.21.15.170:0/3450479149 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f44401224c0 0x7f4440061c50 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:31.104 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.101+0000 7f444591a700 1 -- 172.21.15.170:0/3450479149 >> 172.21.15.170:0/3450479149 conn(0x7f4440091390 msgr2=0x7f4440121920 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:31.104 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.101+0000 7f444591a700 1 -- 172.21.15.170:0/3450479149 shutdown_connections 2020-03-19T21:18:31.104 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.102+0000 7f444591a700 1 -- 172.21.15.170:0/3450479149 wait complete. 2020-03-19T21:18:31.113 INFO:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2020-03-19T21:18:31.221 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.220+0000 7f91417b4700 1 Processor -- start 2020-03-19T21:18:31.221 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.220+0000 7f91417b4700 1 -- start start 2020-03-19T21:18:31.227 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.221+0000 7f91417b4700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f913c125840 0x7f913c127c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T21:18:31.227 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.221+0000 7f91417b4700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f913c1228a0 con 0x7f913c1224c0 2020-03-19T21:18:31.227 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.221+0000 7f91417b4700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f913c092200 con 0x7f913c125840 2020-03-19T21:18:31.227 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.221+0000 7f913a7fc700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f913c125840 0x7f913c127c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:31.227 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.221+0000 7f913affd700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f913c1224c0 0x7f913c122ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:54002/0 (socket says 172.21.15.170:54002) 2020-03-19T21:18:31.227 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.221+0000 7f913affd700 1 -- 172.21.15.170:0/830757747 learned_addr learned my addr 172.21.15.170:0/830757747 (peer_addr_for_me v1:172.21.15.170:0/0) 2020-03-19T21:18:31.228 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.221+0000 7f913a7fc700 1 -- 172.21.15.170:0/830757747 >> v1:172.21.15.170:6789/0 conn(0x7f913c1224c0 legacy=0x7f913c122ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:31.228 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.221+0000 7f913a7fc700 1 -- 172.21.15.170:0/830757747 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f913c090b60 con 0x7f913c125840 2020-03-19T21:18:31.228 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.221+0000 7f9139ffb700 1 -- 172.21.15.170:0/830757747 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2477849364 0 0) 0x7f91300010d0 con 0x7f913c1224c0 2020-03-19T21:18:31.228 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.221+0000 7f913a7fc700 1 --2- 172.21.15.170:0/830757747 >> v2:172.21.15.170:3300/0 conn(0x7f913c125840 0x7f913c127c30 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f912c002f30 tx=0x7f912c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:31.228 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.221+0000 7f913a7fc700 1 --2- 172.21.15.170:0/830757747 >> v2:172.21.15.170:3300/0 conn(0x7f913c125840 0x7f913c127c30 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f912c002f30 tx=0x7f912c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:31.228 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.221+0000 7f913a7fc700 1 --2- 172.21.15.170:0/830757747 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f913c125840 0x7f913c127c30 secure :-1 s=READY pgs=243 cs=0 l=1 rx=0x7f912c002f30 tx=0x7f912c007a40).ready entity=mon.0 client_cookie=fe0ca107e6117b0a server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:31.228 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.221+0000 7f913a7fc700 1 --2- 172.21.15.170:0/830757747 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f913c125840 0x7f913c127c30 secure :-1 s=THROTTLE_DONE pgs=243 cs=0 l=1 rx=0x7f912c002f30 tx=0x7f912c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:31.229 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.221+0000 7f9139ffb700 1 -- 172.21.15.170:0/830757747 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f912c008e80 con 0x7f913c125840 2020-03-19T21:18:31.229 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.221+0000 7f913a7fc700 1 --2- 172.21.15.170:0/830757747 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f913c125840 0x7f913c127c30 secure :-1 s=THROTTLE_DONE pgs=243 cs=0 l=1 rx=0x7f912c002f30 tx=0x7f912c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:31.229 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.221+0000 7f913a7fc700 1 --2- 172.21.15.170:0/830757747 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f913c125840 0x7f913c127c30 secure :-1 s=THROTTLE_DONE pgs=243 cs=0 l=1 rx=0x7f912c002f30 tx=0x7f912c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:31.229 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.221+0000 7f9139ffb700 1 -- 172.21.15.170:0/830757747 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f912c0113f0 con 0x7f913c125840 2020-03-19T21:18:31.229 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.221+0000 7f9139ffb700 1 -- 172.21.15.170:0/830757747 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f912c0116d0 con 0x7f913c125840 2020-03-19T21:18:31.229 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.221+0000 7f91417b4700 1 -- 172.21.15.170:0/830757747 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f913c125840 msgr2=0x7f913c127c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:31.229 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.221+0000 7f91417b4700 1 --2- 172.21.15.170:0/830757747 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f913c125840 0x7f913c127c30 secure :-1 s=READY pgs=243 cs=0 l=1 rx=0x7f912c002f30 tx=0x7f912c007a40).stop 2020-03-19T21:18:31.230 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.222+0000 7f91417b4700 1 -- 172.21.15.170:0/830757747 shutdown_connections 2020-03-19T21:18:31.230 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.222+0000 7f91417b4700 1 --2- 172.21.15.170:0/830757747 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f913c125840 0x7f913c127c30 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:31.230 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.222+0000 7f91417b4700 1 -- 172.21.15.170:0/830757747 >> 172.21.15.170:0/830757747 conn(0x7f913c091390 msgr2=0x7f913c091770 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:31.230 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.222+0000 7f91417b4700 1 -- 172.21.15.170:0/830757747 shutdown_connections 2020-03-19T21:18:31.230 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.222+0000 7f91417b4700 1 -- 172.21.15.170:0/830757747 wait complete. 2020-03-19T21:18:31.230 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.223+0000 7f91417b4700 1 Processor -- start 2020-03-19T21:18:31.230 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.223+0000 7f91417b4700 1 -- start start 2020-03-19T21:18:31.231 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.223+0000 7f91417b4700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f913c125840 0x7f913c10c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:31.231 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.223+0000 7f91417b4700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f913c066380 con 0x7f913c1224c0 2020-03-19T21:18:31.231 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.223+0000 7f91417b4700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f913c08f6b0 con 0x7f913c125840 2020-03-19T21:18:31.231 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.223+0000 7f913affd700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f913c1224c0 0x7f913c10c1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:54006/0 (socket says 172.21.15.170:54006) 2020-03-19T21:18:31.231 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.223+0000 7f913affd700 1 -- 172.21.15.170:0/2861198938 learned_addr learned my addr 172.21.15.170:0/2861198938 (peer_addr_for_me v1:172.21.15.170:0/0) 2020-03-19T21:18:31.231 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.223+0000 7f913a7fc700 1 --2- 172.21.15.170:0/2861198938 >> v2:172.21.15.170:3300/0 conn(0x7f913c125840 0x7f913c10c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:31.231 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.223+0000 7f913a7fc700 1 -- 172.21.15.170:0/2861198938 >> v1:172.21.15.170:6789/0 conn(0x7f913c1224c0 legacy=0x7f913c10c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:31.231 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.223+0000 7f912bfff700 1 -- 172.21.15.170:0/2861198938 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2836607298 0 0) 0x7f913c066380 con 0x7f913c1224c0 2020-03-19T21:18:31.232 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.223+0000 7f913a7fc700 1 -- 172.21.15.170:0/2861198938 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f913c0665b0 con 0x7f913c125840 2020-03-19T21:18:31.232 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.223+0000 7f913a7fc700 1 --2- 172.21.15.170:0/2861198938 >> v2:172.21.15.170:3300/0 conn(0x7f913c125840 0x7f913c10c8b0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f912c002f30 tx=0x7f912c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:31.232 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.223+0000 7f913a7fc700 1 --2- 172.21.15.170:0/2861198938 >> v2:172.21.15.170:3300/0 conn(0x7f913c125840 0x7f913c10c8b0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f912c002f30 tx=0x7f912c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:31.232 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.223+0000 7f913a7fc700 1 --2- 172.21.15.170:0/2861198938 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f913c125840 0x7f913c10c8b0 secure :-1 s=READY pgs=245 cs=0 l=1 rx=0x7f912c002f30 tx=0x7f912c007a40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:31.232 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.224+0000 7f913a7fc700 1 --2- 172.21.15.170:0/2861198938 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f913c125840 0x7f913c10c8b0 secure :-1 s=THROTTLE_DONE pgs=245 cs=0 l=1 rx=0x7f912c002f30 tx=0x7f912c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:31.232 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.224+0000 7f912bfff700 1 -- 172.21.15.170:0/2861198938 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f912c0116c0 con 0x7f913c125840 2020-03-19T21:18:31.232 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.224+0000 7f913a7fc700 1 --2- 172.21.15.170:0/2861198938 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f913c125840 0x7f913c10c8b0 secure :-1 s=THROTTLE_DONE pgs=245 cs=0 l=1 rx=0x7f912c002f30 tx=0x7f912c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:31.233 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.224+0000 7f913a7fc700 1 --2- 172.21.15.170:0/2861198938 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f913c125840 0x7f913c10c8b0 secure :-1 s=THROTTLE_DONE pgs=245 cs=0 l=1 rx=0x7f912c002f30 tx=0x7f912c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:31.233 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.224+0000 7f912bfff700 1 -- 172.21.15.170:0/2861198938 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f912c00f830 con 0x7f913c125840 2020-03-19T21:18:31.233 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.224+0000 7f91417b4700 1 -- 172.21.15.170:0/2861198938 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f913c08fee0 con 0x7f913c125840 2020-03-19T21:18:31.233 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.224+0000 7f912bfff700 1 -- 172.21.15.170:0/2861198938 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f912c007280 con 0x7f913c125840 2020-03-19T21:18:31.233 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.224+0000 7f91417b4700 1 -- 172.21.15.170:0/2861198938 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f913c10af60 con 0x7f913c125840 2020-03-19T21:18:31.233 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.224+0000 7f913a7fc700 1 --2- 172.21.15.170:0/2861198938 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f913c125840 0x7f913c10c8b0 secure :-1 s=THROTTLE_DONE pgs=245 cs=0 l=1 rx=0x7f912c002f30 tx=0x7f912c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:31.234 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.224+0000 7f913a7fc700 1 --2- 172.21.15.170:0/2861198938 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f913c125840 0x7f913c10c8b0 secure :-1 s=THROTTLE_DONE pgs=245 cs=0 l=1 rx=0x7f912c002f30 tx=0x7f912c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:31.234 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.224+0000 7f912bfff700 1 -- 172.21.15.170:0/2861198938 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 4) v1 ==== 29603+0+0 (secure 0 0 0) 0x7f912c007470 con 0x7f913c125840 2020-03-19T21:18:31.234 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.224+0000 7f912bfff700 1 --2- 172.21.15.170:0/2861198938 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f9118028280 0x7f911802a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:31.234 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.224+0000 7f912bfff700 1 -- 172.21.15.170:0/2861198938 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 3786+0+0 (secure 0 0 0) 0x7f912c0345b0 con 0x7f913c125840 2020-03-19T21:18:31.234 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.225+0000 7f913affd700 1 --2- 172.21.15.170:0/2861198938 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f9118028280 0x7f911802a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:31.234 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.225+0000 7f91417b4700 1 -- 172.21.15.170:0/2861198938 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f913c090b60 con 0x7f913c125840 2020-03-19T21:18:31.234 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.225+0000 7f913affd700 1 --2- 172.21.15.170:0/2861198938 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f9118028280 0x7f911802a740 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f9130005500 tx=0x7f91300016e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:31.235 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.225+0000 7f913affd700 1 --2- 172.21.15.170:0/2861198938 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f9118028280 0x7f911802a740 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f9130005500 tx=0x7f91300016e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:31.235 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.225+0000 7f913affd700 1 --2- 172.21.15.170:0/2861198938 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f9118028280 0x7f911802a740 secure :-1 s=READY pgs=61 cs=0 l=1 rx=0x7f9130005500 tx=0x7f91300016e0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:31.235 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.228+0000 7f913a7fc700 1 --2- 172.21.15.170:0/2861198938 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f913c125840 0x7f913c10c8b0 secure :-1 s=THROTTLE_DONE pgs=245 cs=0 l=1 rx=0x7f912c002f30 tx=0x7f912c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:31.235 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.228+0000 7f912bfff700 1 -- 172.21.15.170:0/2861198938 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+126772 (secure 0 0 0) 0x7f913c090b60 con 0x7f913c125840 2020-03-19T21:18:31.368 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.367+0000 7f91417b4700 1 -- 172.21.15.170:0/2861198938 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f913c090b60 con 0x7f913c125840 2020-03-19T21:18:31.369 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.367+0000 7f913a7fc700 1 --2- 172.21.15.170:0/2861198938 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f913c125840 0x7f913c10c8b0 secure :-1 s=THROTTLE_DONE pgs=245 cs=0 l=1 rx=0x7f912c002f30 tx=0x7f912c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:31.369 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.367+0000 7f912bfff700 1 -- 172.21.15.170:0/2861198938 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+10 (secure 0 0 0) 0x7f912c014020 con 0x7f913c125840 2020-03-19T21:18:31.369 INFO:teuthology.orchestra.run.smithi170.stdout:pg_num: 8 2020-03-19T21:18:31.370 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.369+0000 7f91417b4700 1 -- 172.21.15.170:0/2861198938 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f9118028280 msgr2=0x7f911802a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:31.370 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.369+0000 7f91417b4700 1 --2- 172.21.15.170:0/2861198938 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f9118028280 0x7f911802a740 secure :-1 s=READY pgs=61 cs=0 l=1 rx=0x7f9130005500 tx=0x7f91300016e0).stop 2020-03-19T21:18:31.371 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.369+0000 7f91417b4700 1 -- 172.21.15.170:0/2861198938 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f913c125840 msgr2=0x7f913c10c8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:31.371 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.369+0000 7f91417b4700 1 --2- 172.21.15.170:0/2861198938 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f913c125840 0x7f913c10c8b0 secure :-1 s=READY pgs=245 cs=0 l=1 rx=0x7f912c002f30 tx=0x7f912c007a40).stop 2020-03-19T21:18:31.371 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.369+0000 7f91417b4700 1 -- 172.21.15.170:0/2861198938 shutdown_connections 2020-03-19T21:18:31.371 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.369+0000 7f91417b4700 1 --2- 172.21.15.170:0/2861198938 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f9118028280 0x7f911802a740 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:31.371 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.369+0000 7f91417b4700 1 --2- 172.21.15.170:0/2861198938 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f913c125840 0x7f913c10c8b0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:31.371 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.369+0000 7f91417b4700 1 -- 172.21.15.170:0/2861198938 >> 172.21.15.170:0/2861198938 conn(0x7f913c091390 msgr2=0x7f913c124800 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:31.371 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.369+0000 7f91417b4700 1 -- 172.21.15.170:0/2861198938 shutdown_connections 2020-03-19T21:18:31.372 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.369+0000 7f91417b4700 1 -- 172.21.15.170:0/2861198938 wait complete. 2020-03-19T21:18:31.380 INFO:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get data_cache pg_num 2020-03-19T21:18:31.487 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.486+0000 7fb46148f700 1 Processor -- start 2020-03-19T21:18:31.487 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.486+0000 7fb46148f700 1 -- start start 2020-03-19T21:18:31.492 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.487+0000 7fb46148f700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fb45c125840 0x7fb45c127c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T21:18:31.492 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.487+0000 7fb46148f700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb45c1228a0 con 0x7fb45c1224c0 2020-03-19T21:18:31.493 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.487+0000 7fb46148f700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb45c092200 con 0x7fb45c125840 2020-03-19T21:18:31.493 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.487+0000 7fb45a7fc700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fb45c125840 0x7fb45c127c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:31.493 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.487+0000 7fb45a7fc700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fb45c125840 0x7fb45c127c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53250/0 (socket says 172.21.15.170:53250) 2020-03-19T21:18:31.493 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.487+0000 7fb45a7fc700 1 -- 172.21.15.170:0/750491320 learned_addr learned my addr 172.21.15.170:0/750491320 (peer_addr_for_me v2:172.21.15.170:0/0) 2020-03-19T21:18:31.493 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.487+0000 7fb45affd700 1 --1- 172.21.15.170:0/750491320 >> v1:172.21.15.170:6789/0 conn(0x7fb45c1224c0 0x7fb45c122ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:54012/0 (socket says 172.21.15.170:54012) 2020-03-19T21:18:31.493 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.487+0000 7fb45a7fc700 1 -- 172.21.15.170:0/750491320 >> v1:172.21.15.170:6789/0 conn(0x7fb45c1224c0 legacy=0x7fb45c122ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:31.494 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.487+0000 7fb459ffb700 1 -- 172.21.15.170:0/750491320 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4090690685 0 0) 0x7fb4500010d0 con 0x7fb45c1224c0 2020-03-19T21:18:31.494 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.487+0000 7fb45a7fc700 1 -- 172.21.15.170:0/750491320 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb45c090b60 con 0x7fb45c125840 2020-03-19T21:18:31.494 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.487+0000 7fb45a7fc700 1 --2- 172.21.15.170:0/750491320 >> v2:172.21.15.170:3300/0 conn(0x7fb45c125840 0x7fb45c127c30 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7fb44c002f30 tx=0x7fb44c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:31.494 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.487+0000 7fb45a7fc700 1 --2- 172.21.15.170:0/750491320 >> v2:172.21.15.170:3300/0 conn(0x7fb45c125840 0x7fb45c127c30 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7fb44c002f30 tx=0x7fb44c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:31.494 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.487+0000 7fb45a7fc700 1 --2- 172.21.15.170:0/750491320 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb45c125840 0x7fb45c127c30 secure :-1 s=READY pgs=247 cs=0 l=1 rx=0x7fb44c002f30 tx=0x7fb44c007a40).ready entity=mon.0 client_cookie=60473f36c9f2d441 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:31.494 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.487+0000 7fb45a7fc700 1 --2- 172.21.15.170:0/750491320 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb45c125840 0x7fb45c127c30 secure :-1 s=THROTTLE_DONE pgs=247 cs=0 l=1 rx=0x7fb44c002f30 tx=0x7fb44c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:31.494 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.487+0000 7fb459ffb700 1 -- 172.21.15.170:0/750491320 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fb44c0114b0 con 0x7fb45c125840 2020-03-19T21:18:31.494 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.487+0000 7fb45a7fc700 1 --2- 172.21.15.170:0/750491320 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb45c125840 0x7fb45c127c30 secure :-1 s=THROTTLE_DONE pgs=247 cs=0 l=1 rx=0x7fb44c002f30 tx=0x7fb44c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:31.495 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.487+0000 7fb45a7fc700 1 --2- 172.21.15.170:0/750491320 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb45c125840 0x7fb45c127c30 secure :-1 s=THROTTLE_DONE pgs=247 cs=0 l=1 rx=0x7fb44c002f30 tx=0x7fb44c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:31.495 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.487+0000 7fb459ffb700 1 -- 172.21.15.170:0/750491320 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb44c011610 con 0x7fb45c125840 2020-03-19T21:18:31.495 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.487+0000 7fb459ffb700 1 -- 172.21.15.170:0/750491320 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fb44c011930 con 0x7fb45c125840 2020-03-19T21:18:31.495 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.487+0000 7fb46148f700 1 -- 172.21.15.170:0/750491320 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb45c125840 msgr2=0x7fb45c127c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:31.495 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.487+0000 7fb46148f700 1 --2- 172.21.15.170:0/750491320 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb45c125840 0x7fb45c127c30 secure :-1 s=READY pgs=247 cs=0 l=1 rx=0x7fb44c002f30 tx=0x7fb44c007a40).stop 2020-03-19T21:18:31.495 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.488+0000 7fb46148f700 1 -- 172.21.15.170:0/750491320 shutdown_connections 2020-03-19T21:18:31.495 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.488+0000 7fb46148f700 1 --2- 172.21.15.170:0/750491320 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb45c125840 0x7fb45c127c30 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:31.496 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.488+0000 7fb46148f700 1 -- 172.21.15.170:0/750491320 >> 172.21.15.170:0/750491320 conn(0x7fb45c091390 msgr2=0x7fb45c091770 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:31.496 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.488+0000 7fb46148f700 1 -- 172.21.15.170:0/750491320 shutdown_connections 2020-03-19T21:18:31.496 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.488+0000 7fb46148f700 1 -- 172.21.15.170:0/750491320 wait complete. 2020-03-19T21:18:31.496 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.489+0000 7fb46148f700 1 Processor -- start 2020-03-19T21:18:31.496 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.489+0000 7fb46148f700 1 -- start start 2020-03-19T21:18:31.496 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.489+0000 7fb46148f700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fb45c1224c0 0x7fb45c08f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:31.496 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.489+0000 7fb46148f700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb45c11ee00 con 0x7fb45c125840 2020-03-19T21:18:31.497 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.489+0000 7fb46148f700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb45c10b460 con 0x7fb45c1224c0 2020-03-19T21:18:31.497 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.489+0000 7fb45affd700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fb45c1224c0 0x7fb45c08f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:31.497 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.489+0000 7fb45a7fc700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fb45c125840 0x7fb45c08f620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:54018/0 (socket says 172.21.15.170:54018) 2020-03-19T21:18:31.497 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.489+0000 7fb45a7fc700 1 -- 172.21.15.170:0/256718479 learned_addr learned my addr 172.21.15.170:0/256718479 (peer_addr_for_me v1:172.21.15.170:0/0) 2020-03-19T21:18:31.497 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.489+0000 7fb45affd700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fb45c1224c0 0x7fb45c08f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53252/0 (socket says 172.21.15.170:53252) 2020-03-19T21:18:31.497 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.489+0000 7fb45affd700 1 -- 172.21.15.170:0/256718479 >> v1:172.21.15.170:6789/0 conn(0x7fb45c125840 legacy=0x7fb45c08f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:31.497 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.489+0000 7fb44bfff700 1 -- 172.21.15.170:0/256718479 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1862857292 0 0) 0x7fb45c11ee00 con 0x7fb45c125840 2020-03-19T21:18:31.498 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.489+0000 7fb45affd700 1 -- 172.21.15.170:0/256718479 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb45c066950 con 0x7fb45c1224c0 2020-03-19T21:18:31.498 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.489+0000 7fb45affd700 1 --2- 172.21.15.170:0/256718479 >> v2:172.21.15.170:3300/0 conn(0x7fb45c1224c0 0x7fb45c08f1c0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7fb45c10b3b0 tx=0x7fb4500108d0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:31.498 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.489+0000 7fb45affd700 1 --2- 172.21.15.170:0/256718479 >> v2:172.21.15.170:3300/0 conn(0x7fb45c1224c0 0x7fb45c08f1c0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7fb45c10b3b0 tx=0x7fb4500108d0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:31.498 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.489+0000 7fb45affd700 1 --2- 172.21.15.170:0/256718479 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb45c1224c0 0x7fb45c08f1c0 secure :-1 s=READY pgs=249 cs=0 l=1 rx=0x7fb45c10b3b0 tx=0x7fb4500108d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:31.498 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.490+0000 7fb45affd700 1 --2- 172.21.15.170:0/256718479 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb45c1224c0 0x7fb45c08f1c0 secure :-1 s=THROTTLE_DONE pgs=249 cs=0 l=1 rx=0x7fb45c10b3b0 tx=0x7fb4500108d0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:31.498 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.490+0000 7fb44bfff700 1 -- 172.21.15.170:0/256718479 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fb450011420 con 0x7fb45c1224c0 2020-03-19T21:18:31.498 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.490+0000 7fb45affd700 1 --2- 172.21.15.170:0/256718479 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb45c1224c0 0x7fb45c08f1c0 secure :-1 s=THROTTLE_DONE pgs=249 cs=0 l=1 rx=0x7fb45c10b3b0 tx=0x7fb4500108d0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:31.499 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.490+0000 7fb45affd700 1 --2- 172.21.15.170:0/256718479 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb45c1224c0 0x7fb45c08f1c0 secure :-1 s=THROTTLE_DONE pgs=249 cs=0 l=1 rx=0x7fb45c10b3b0 tx=0x7fb4500108d0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:31.499 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.490+0000 7fb44bfff700 1 -- 172.21.15.170:0/256718479 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4500115c0 con 0x7fb45c1224c0 2020-03-19T21:18:31.499 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.490+0000 7fb46148f700 1 -- 172.21.15.170:0/256718479 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb45c10b770 con 0x7fb45c1224c0 2020-03-19T21:18:31.499 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.490+0000 7fb44bfff700 1 -- 172.21.15.170:0/256718479 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fb4500118e0 con 0x7fb45c1224c0 2020-03-19T21:18:31.499 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.490+0000 7fb46148f700 1 -- 172.21.15.170:0/256718479 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb45c10c170 con 0x7fb45c1224c0 2020-03-19T21:18:31.499 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.490+0000 7fb45affd700 1 --2- 172.21.15.170:0/256718479 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb45c1224c0 0x7fb45c08f1c0 secure :-1 s=THROTTLE_DONE pgs=249 cs=0 l=1 rx=0x7fb45c10b3b0 tx=0x7fb4500108d0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:31.499 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.490+0000 7fb44bfff700 1 -- 172.21.15.170:0/256718479 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 4) v1 ==== 29603+0+0 (secure 0 0 0) 0x7fb45000b0a0 con 0x7fb45c1224c0 2020-03-19T21:18:31.499 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.490+0000 7fb45affd700 1 --2- 172.21.15.170:0/256718479 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb45c1224c0 0x7fb45c08f1c0 secure :-1 s=THROTTLE_DONE pgs=249 cs=0 l=1 rx=0x7fb45c10b3b0 tx=0x7fb4500108d0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:31.500 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.490+0000 7fb44bfff700 1 --2- 172.21.15.170:0/256718479 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7fb438028280 0x7fb43802a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:31.500 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.490+0000 7fb44bfff700 1 -- 172.21.15.170:0/256718479 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 3786+0+0 (secure 0 0 0) 0x7fb450033e30 con 0x7fb45c1224c0 2020-03-19T21:18:31.500 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.490+0000 7fb45a7fc700 1 --2- 172.21.15.170:0/256718479 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7fb438028280 0x7fb43802a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:31.500 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.491+0000 7fb45a7fc700 1 --2- 172.21.15.170:0/256718479 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7fb438028280 0x7fb43802a740 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7fb44c010280 tx=0x7fb44c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:31.500 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.491+0000 7fb46148f700 1 -- 172.21.15.170:0/256718479 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb45c056db0 con 0x7fb45c1224c0 2020-03-19T21:18:31.500 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.491+0000 7fb45a7fc700 1 --2- 172.21.15.170:0/256718479 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7fb438028280 0x7fb43802a740 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7fb44c010280 tx=0x7fb44c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:31.500 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.491+0000 7fb45a7fc700 1 --2- 172.21.15.170:0/256718479 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7fb438028280 0x7fb43802a740 secure :-1 s=READY pgs=62 cs=0 l=1 rx=0x7fb44c010280 tx=0x7fb44c007a40).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:31.501 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.494+0000 7fb45affd700 1 --2- 172.21.15.170:0/256718479 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb45c1224c0 0x7fb45c08f1c0 secure :-1 s=THROTTLE_DONE pgs=249 cs=0 l=1 rx=0x7fb45c10b3b0 tx=0x7fb4500108d0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:31.501 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.494+0000 7fb44bfff700 1 -- 172.21.15.170:0/256718479 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+126772 (secure 0 0 0) 0x7fb450007160 con 0x7fb45c1224c0 2020-03-19T21:18:31.632 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.631+0000 7fb46148f700 1 -- 172.21.15.170:0/256718479 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "data_cache", "var": "pg_num"} v 0) v1 -- 0x7fb45c08ed10 con 0x7fb45c1224c0 2020-03-19T21:18:31.633 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.631+0000 7fb45affd700 1 --2- 172.21.15.170:0/256718479 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb45c1224c0 0x7fb45c08f1c0 secure :-1 s=THROTTLE_DONE pgs=249 cs=0 l=1 rx=0x7fb45c10b3b0 tx=0x7fb4500108d0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:31.633 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.631+0000 7fb44bfff700 1 -- 172.21.15.170:0/256718479 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "data_cache", "var": "pg_num"}]=0 v24) v1 ==== 99+0+10 (secure 0 0 0) 0x7fb450007340 con 0x7fb45c1224c0 2020-03-19T21:18:31.633 INFO:teuthology.orchestra.run.smithi170.stdout:pg_num: 4 2020-03-19T21:18:31.634 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.633+0000 7fb46148f700 1 -- 172.21.15.170:0/256718479 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7fb438028280 msgr2=0x7fb43802a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:31.634 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.633+0000 7fb46148f700 1 --2- 172.21.15.170:0/256718479 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7fb438028280 0x7fb43802a740 secure :-1 s=READY pgs=62 cs=0 l=1 rx=0x7fb44c010280 tx=0x7fb44c007a40).stop 2020-03-19T21:18:31.634 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.633+0000 7fb46148f700 1 -- 172.21.15.170:0/256718479 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb45c1224c0 msgr2=0x7fb45c08f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:31.635 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.633+0000 7fb46148f700 1 --2- 172.21.15.170:0/256718479 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb45c1224c0 0x7fb45c08f1c0 secure :-1 s=READY pgs=249 cs=0 l=1 rx=0x7fb45c10b3b0 tx=0x7fb4500108d0).stop 2020-03-19T21:18:31.635 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.633+0000 7fb46148f700 1 -- 172.21.15.170:0/256718479 shutdown_connections 2020-03-19T21:18:31.635 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.633+0000 7fb46148f700 1 --2- 172.21.15.170:0/256718479 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7fb438028280 0x7fb43802a740 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:31.635 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.633+0000 7fb46148f700 1 --2- 172.21.15.170:0/256718479 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb45c1224c0 0x7fb45c08f1c0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:31.635 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.633+0000 7fb46148f700 1 -- 172.21.15.170:0/256718479 >> 172.21.15.170:0/256718479 conn(0x7fb45c091390 msgr2=0x7fb45c11f220 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:31.635 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.633+0000 7fb46148f700 1 -- 172.21.15.170:0/256718479 shutdown_connections 2020-03-19T21:18:31.635 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.633+0000 7fb46148f700 1 -- 172.21.15.170:0/256718479 wait complete. 2020-03-19T21:18:31.644 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-03-19T21:18:31.644 INFO:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-03-19T21:18:31.749 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.748+0000 7f60f3ed9700 1 Processor -- start 2020-03-19T21:18:31.749 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.748+0000 7f60f3ed9700 1 -- start start 2020-03-19T21:18:31.755 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.749+0000 7f60f3ed9700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f60ec1221d0 0x7f60ec1225b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T21:18:31.755 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.749+0000 7f60f3ed9700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60ec08eee0 con 0x7f60ec122a10 2020-03-19T21:18:31.755 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.749+0000 7f60f3ed9700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60ec08dda0 con 0x7f60ec1221d0 2020-03-19T21:18:31.755 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.749+0000 7f60f1c75700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f60ec1221d0 0x7f60ec1225b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:31.755 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.749+0000 7f60f1c75700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f60ec1221d0 0x7f60ec1225b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53258/0 (socket says 172.21.15.170:53258) 2020-03-19T21:18:31.756 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.749+0000 7f60f1c75700 1 -- 172.21.15.170:0/559718585 learned_addr learned my addr 172.21.15.170:0/559718585 (peer_addr_for_me v2:172.21.15.170:0/0) 2020-03-19T21:18:31.756 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.749+0000 7f60f1474700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f60ec122a10 0x7f60ec08bef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:54022/0 (socket says 172.21.15.170:54022) 2020-03-19T21:18:31.756 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.749+0000 7f60f1c75700 1 -- 172.21.15.170:0/559718585 >> v1:172.21.15.170:6789/0 conn(0x7f60ec122a10 legacy=0x7f60ec08bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:31.756 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.749+0000 7f60f1c75700 1 -- 172.21.15.170:0/559718585 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60ec121fd0 con 0x7f60ec1221d0 2020-03-19T21:18:31.756 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.749+0000 7f60f0c73700 1 -- 172.21.15.170:0/559718585 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2068678488 0 0) 0x7f60ec08eee0 con 0x7f60ec122a10 2020-03-19T21:18:31.756 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.749+0000 7f60f1c75700 1 --2- 172.21.15.170:0/559718585 >> v2:172.21.15.170:3300/0 conn(0x7f60ec1221d0 0x7f60ec1225b0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f60dc002f30 tx=0x7f60dc007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:31.756 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.749+0000 7f60f1c75700 1 --2- 172.21.15.170:0/559718585 >> v2:172.21.15.170:3300/0 conn(0x7f60ec1221d0 0x7f60ec1225b0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f60dc002f30 tx=0x7f60dc007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:31.757 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.749+0000 7f60f1c75700 1 --2- 172.21.15.170:0/559718585 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f60ec1221d0 0x7f60ec1225b0 secure :-1 s=READY pgs=251 cs=0 l=1 rx=0x7f60dc002f30 tx=0x7f60dc007a40).ready entity=mon.0 client_cookie=877df59f1f1bb37c server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:31.757 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.749+0000 7f60f1c75700 1 --2- 172.21.15.170:0/559718585 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f60ec1221d0 0x7f60ec1225b0 secure :-1 s=THROTTLE_DONE pgs=251 cs=0 l=1 rx=0x7f60dc002f30 tx=0x7f60dc007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:31.757 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.749+0000 7f60f0c73700 1 -- 172.21.15.170:0/559718585 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f60dc0114b0 con 0x7f60ec1221d0 2020-03-19T21:18:31.757 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.749+0000 7f60f1c75700 1 --2- 172.21.15.170:0/559718585 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f60ec1221d0 0x7f60ec1225b0 secure :-1 s=THROTTLE_DONE pgs=251 cs=0 l=1 rx=0x7f60dc002f30 tx=0x7f60dc007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:31.757 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.749+0000 7f60f0c73700 1 -- 172.21.15.170:0/559718585 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60dc011610 con 0x7f60ec1221d0 2020-03-19T21:18:31.757 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.749+0000 7f60f1c75700 1 --2- 172.21.15.170:0/559718585 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f60ec1221d0 0x7f60ec1225b0 secure :-1 s=THROTTLE_DONE pgs=251 cs=0 l=1 rx=0x7f60dc002f30 tx=0x7f60dc007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:31.757 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.749+0000 7f60f0c73700 1 -- 172.21.15.170:0/559718585 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f60dc0118f0 con 0x7f60ec1221d0 2020-03-19T21:18:31.758 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.749+0000 7f60f3ed9700 1 -- 172.21.15.170:0/559718585 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f60ec1221d0 msgr2=0x7f60ec1225b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:31.758 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.749+0000 7f60f3ed9700 1 --2- 172.21.15.170:0/559718585 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f60ec1221d0 0x7f60ec1225b0 secure :-1 s=READY pgs=251 cs=0 l=1 rx=0x7f60dc002f30 tx=0x7f60dc007a40).stop 2020-03-19T21:18:31.758 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.750+0000 7f60f3ed9700 1 -- 172.21.15.170:0/559718585 shutdown_connections 2020-03-19T21:18:31.758 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.750+0000 7f60f3ed9700 1 --2- 172.21.15.170:0/559718585 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f60ec1221d0 0x7f60ec1225b0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:31.758 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.750+0000 7f60f3ed9700 1 -- 172.21.15.170:0/559718585 >> 172.21.15.170:0/559718585 conn(0x7f60ec091540 msgr2=0x7f60ec091950 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:31.758 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.750+0000 7f60f3ed9700 1 -- 172.21.15.170:0/559718585 shutdown_connections 2020-03-19T21:18:31.758 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.750+0000 7f60f3ed9700 1 -- 172.21.15.170:0/559718585 wait complete. 2020-03-19T21:18:31.759 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.750+0000 7f60f3ed9700 1 Processor -- start 2020-03-19T21:18:31.759 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.750+0000 7f60f3ed9700 1 -- start start 2020-03-19T21:18:31.759 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.751+0000 7f60f3ed9700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f60ec1221d0 0x7f60ec11b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:31.759 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.751+0000 7f60f3ed9700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60ec11f980 con 0x7f60ec122a10 2020-03-19T21:18:31.759 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.751+0000 7f60f3ed9700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60ec11ed80 con 0x7f60ec1221d0 2020-03-19T21:18:31.759 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.751+0000 7f60f1c75700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f60ec1221d0 0x7f60ec11b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:31.759 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.751+0000 7f60f1c75700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f60ec1221d0 0x7f60ec11b570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53262/0 (socket says 172.21.15.170:53262) 2020-03-19T21:18:31.760 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.751+0000 7f60f1c75700 1 -- 172.21.15.170:0/2676328229 learned_addr learned my addr 172.21.15.170:0/2676328229 (peer_addr_for_me v2:172.21.15.170:0/0) 2020-03-19T21:18:31.760 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.751+0000 7f60f1474700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f60ec122a10 0x7f60ec11f270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:54028/0 (socket says 172.21.15.170:54028) 2020-03-19T21:18:31.760 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.751+0000 7f60f1c75700 1 -- 172.21.15.170:0/2676328229 >> v1:172.21.15.170:6789/0 conn(0x7f60ec122a10 legacy=0x7f60ec11f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:31.760 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.751+0000 7f60f1c75700 1 -- 172.21.15.170:0/2676328229 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60ec11ef60 con 0x7f60ec1221d0 2020-03-19T21:18:31.760 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.751+0000 7f60daffd700 1 -- 172.21.15.170:0/2676328229 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4077073249 0 0) 0x7f60ec11f980 con 0x7f60ec122a10 2020-03-19T21:18:31.760 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.751+0000 7f60f1c75700 1 --2- 172.21.15.170:0/2676328229 >> v2:172.21.15.170:3300/0 conn(0x7f60ec1221d0 0x7f60ec11b570 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f60ec11c370 tx=0x7f60dc007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:31.760 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.751+0000 7f60f1c75700 1 --2- 172.21.15.170:0/2676328229 >> v2:172.21.15.170:3300/0 conn(0x7f60ec1221d0 0x7f60ec11b570 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f60ec11c370 tx=0x7f60dc007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:31.761 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.751+0000 7f60f1c75700 1 --2- 172.21.15.170:0/2676328229 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f60ec1221d0 0x7f60ec11b570 secure :-1 s=READY pgs=253 cs=0 l=1 rx=0x7f60ec11c370 tx=0x7f60dc007a40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:31.761 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.751+0000 7f60f1c75700 1 --2- 172.21.15.170:0/2676328229 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f60ec1221d0 0x7f60ec11b570 secure :-1 s=THROTTLE_DONE pgs=253 cs=0 l=1 rx=0x7f60ec11c370 tx=0x7f60dc007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:31.761 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.751+0000 7f60daffd700 1 -- 172.21.15.170:0/2676328229 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f60dc00f8e0 con 0x7f60ec1221d0 2020-03-19T21:18:31.761 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.751+0000 7f60f1c75700 1 --2- 172.21.15.170:0/2676328229 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f60ec1221d0 0x7f60ec11b570 secure :-1 s=THROTTLE_DONE pgs=253 cs=0 l=1 rx=0x7f60ec11c370 tx=0x7f60dc007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:31.761 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.751+0000 7f60f1c75700 1 --2- 172.21.15.170:0/2676328229 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f60ec1221d0 0x7f60ec11b570 secure :-1 s=THROTTLE_DONE pgs=253 cs=0 l=1 rx=0x7f60ec11c370 tx=0x7f60dc007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:31.761 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.751+0000 7f60daffd700 1 -- 172.21.15.170:0/2676328229 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60dc007150 con 0x7f60ec1221d0 2020-03-19T21:18:31.761 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.751+0000 7f60f3ed9700 1 -- 172.21.15.170:0/2676328229 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60ec11bff0 con 0x7f60ec1221d0 2020-03-19T21:18:31.762 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.752+0000 7f60daffd700 1 -- 172.21.15.170:0/2676328229 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f60dc0073f0 con 0x7f60ec1221d0 2020-03-19T21:18:31.762 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.752+0000 7f60f3ed9700 1 -- 172.21.15.170:0/2676328229 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60ec11cbb0 con 0x7f60ec1221d0 2020-03-19T21:18:31.762 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.752+0000 7f60f1c75700 1 --2- 172.21.15.170:0/2676328229 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f60ec1221d0 0x7f60ec11b570 secure :-1 s=THROTTLE_DONE pgs=253 cs=0 l=1 rx=0x7f60ec11c370 tx=0x7f60dc007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:31.762 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.752+0000 7f60daffd700 1 -- 172.21.15.170:0/2676328229 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 4) v1 ==== 29603+0+0 (secure 0 0 0) 0x7f60dc0056c0 con 0x7f60ec1221d0 2020-03-19T21:18:31.762 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.752+0000 7f60f1c75700 1 --2- 172.21.15.170:0/2676328229 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f60ec1221d0 0x7f60ec11b570 secure :-1 s=THROTTLE_DONE pgs=253 cs=0 l=1 rx=0x7f60ec11c370 tx=0x7f60dc007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:31.762 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.752+0000 7f60daffd700 1 --2- 172.21.15.170:0/2676328229 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f60cc028250 0x7f60cc02a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:31.762 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.752+0000 7f60daffd700 1 -- 172.21.15.170:0/2676328229 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 3786+0+0 (secure 0 0 0) 0x7f60dc034930 con 0x7f60ec1221d0 2020-03-19T21:18:31.762 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.753+0000 7f60f1474700 1 --2- 172.21.15.170:0/2676328229 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f60cc028250 0x7f60cc02a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:31.763 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.753+0000 7f60f1474700 1 --2- 172.21.15.170:0/2676328229 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f60cc028250 0x7f60cc02a710 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f60e8001110 tx=0x7f60e8001740).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:31.763 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.753+0000 7f60f3ed9700 1 -- 172.21.15.170:0/2676328229 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60ec11e780 con 0x7f60ec1221d0 2020-03-19T21:18:31.763 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.753+0000 7f60f1474700 1 --2- 172.21.15.170:0/2676328229 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f60cc028250 0x7f60cc02a710 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f60e8001110 tx=0x7f60e8001740).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:31.763 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.753+0000 7f60f1474700 1 --2- 172.21.15.170:0/2676328229 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f60cc028250 0x7f60cc02a710 secure :-1 s=READY pgs=63 cs=0 l=1 rx=0x7f60e8001110 tx=0x7f60e8001740).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:31.763 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.756+0000 7f60f1c75700 1 --2- 172.21.15.170:0/2676328229 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f60ec1221d0 0x7f60ec11b570 secure :-1 s=THROTTLE_DONE pgs=253 cs=0 l=1 rx=0x7f60ec11c370 tx=0x7f60dc007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:31.764 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.756+0000 7f60daffd700 1 -- 172.21.15.170:0/2676328229 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+126772 (secure 0 0 0) 0x7f60dc009de0 con 0x7f60ec1221d0 2020-03-19T21:18:31.939 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.937+0000 7f60f3ed9700 1 -- 172.21.15.170:0/2676328229 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f60ec056db0 con 0x7f60ec1221d0 2020-03-19T21:18:31.939 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.938+0000 7f60f1c75700 1 --2- 172.21.15.170:0/2676328229 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f60ec1221d0 0x7f60ec11b570 secure :-1 s=THROTTLE_DONE pgs=253 cs=0 l=1 rx=0x7f60ec11c370 tx=0x7f60dc007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:31.939 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.938+0000 7f60daffd700 1 -- 172.21.15.170:0/2676328229 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+1786 (secure 0 0 0) 0x7f60dc009de0 con 0x7f60ec1221d0 2020-03-19T21:18:31.939 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:18:31.940 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-03-19T21:18:13.514585+0000","modified":"2020-03-19T21:18:17.196186+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4202},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4202":{"gid":4202,"name":"a","rank":0,"incarnation":5,"state":"up:active","state_seq":2,"addr":"172.21.15.170:6827/2695458498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":2695458498},{"type":"v1","addr":"172.21.15.170:6827","nonce":2695458498}]},"join_fscid":-1,"export_targets":[],"features":4540138292836696063,"flags":0}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2020-03-19T21:18:31.941 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.939+0000 7f60f3ed9700 1 -- 172.21.15.170:0/2676328229 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f60cc028250 msgr2=0x7f60cc02a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:31.941 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.939+0000 7f60f3ed9700 1 --2- 172.21.15.170:0/2676328229 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f60cc028250 0x7f60cc02a710 secure :-1 s=READY pgs=63 cs=0 l=1 rx=0x7f60e8001110 tx=0x7f60e8001740).stop 2020-03-19T21:18:31.941 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.939+0000 7f60f3ed9700 1 -- 172.21.15.170:0/2676328229 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f60ec1221d0 msgr2=0x7f60ec11b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:31.941 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.939+0000 7f60f3ed9700 1 --2- 172.21.15.170:0/2676328229 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f60ec1221d0 0x7f60ec11b570 secure :-1 s=READY pgs=253 cs=0 l=1 rx=0x7f60ec11c370 tx=0x7f60dc007a40).stop 2020-03-19T21:18:31.941 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.939+0000 7f60f3ed9700 1 -- 172.21.15.170:0/2676328229 shutdown_connections 2020-03-19T21:18:31.942 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.939+0000 7f60f3ed9700 1 --2- 172.21.15.170:0/2676328229 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f60cc028250 0x7f60cc02a710 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:31.942 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.939+0000 7f60f3ed9700 1 --2- 172.21.15.170:0/2676328229 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f60ec1221d0 0x7f60ec11b570 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:31.942 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.939+0000 7f60f3ed9700 1 -- 172.21.15.170:0/2676328229 >> 172.21.15.170:0/2676328229 conn(0x7f60ec091540 msgr2=0x7f60ec092c70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:31.942 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.939+0000 7f60f3ed9700 1 -- 172.21.15.170:0/2676328229 shutdown_connections 2020-03-19T21:18:31.942 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:31.939+0000 7f60f3ed9700 1 -- 172.21.15.170:0/2676328229 wait complete. 2020-03-19T21:18:31.942 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 6 2020-03-19T21:18:32.051 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {u'session_autoclose': 300, u'balancer': u'', u'up': {u'mds_0': 4202}, u'last_failure_osd_epoch': 0, u'in': [0], u'last_failure': 0, u'max_file_size': 1099511627776, u'explicitly_allowed_features': 0, u'damaged': [], u'tableserver': 0, u'metadata_pool': 2, u'failed': [], u'epoch': 6, u'flags': 18, u'max_mds': 1, u'compat': {u'compat': {}, u'ro_compat': {}, u'incompat': {u'feature_10': u'snaprealm v2', u'feature_8': u'no anchor table', u'feature_9': u'file layout v2', u'feature_2': u'client writeable ranges', u'feature_3': u'default file layouts on dirs', u'feature_1': u'base v0.20', u'feature_6': u'dirfrag is stored in omap', u'feature_4': u'dir inode in separate object', u'feature_5': u'mds uses versioned encoding'}}, u'min_compat_client': u'0 (unknown)', u'data_pools': [3], u'info': {u'gid_4202': {u'export_targets': [], u'name': u'a', u'incarnation': 5, u'state_seq': 2, u'state': u'up:active', u'gid': 4202, u'features': 4540138292836696063, u'rank': 0, u'flags': 0, u'join_fscid': -1, u'addrs': {u'addrvec': [{u'nonce': 2695458498, u'type': u'v2', u'addr': u'172.21.15.170:6826'}, {u'nonce': 2695458498, u'type': u'v1', u'addr': u'172.21.15.170:6827'}]}, u'addr': u'172.21.15.170:6827/2695458498'}}, u'fs_name': u'cephfs', u'created': u'2020-03-19T21:18:13.514585+0000', u'standby_count_wanted': 0, u'enabled': True, u'modified': u'2020-03-19T21:18:17.196186+0000', u'session_timeout': 60, u'stopped': [], u'ever_allowed_features': 0, u'root': 0} 2020-03-19T21:18:32.051 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-03-19T21:18:32.051 INFO:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok status 2020-03-19T21:18:32.168 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2020-03-19T21:18:32.168 INFO:teuthology.orchestra.run.smithi170.stdout: "cluster_fsid": "12d0a33b-6b40-4398-90b4-6b5df1da388e", 2020-03-19T21:18:32.168 INFO:teuthology.orchestra.run.smithi170.stdout: "whoami": 0, 2020-03-19T21:18:32.169 INFO:teuthology.orchestra.run.smithi170.stdout: "id": 4202, 2020-03-19T21:18:32.169 INFO:teuthology.orchestra.run.smithi170.stdout: "want_state": "up:active", 2020-03-19T21:18:32.169 INFO:teuthology.orchestra.run.smithi170.stdout: "state": "up:active", 2020-03-19T21:18:32.169 INFO:teuthology.orchestra.run.smithi170.stdout: "rank_uptime": 15.97159549, 2020-03-19T21:18:32.169 INFO:teuthology.orchestra.run.smithi170.stdout: "mdsmap_epoch": 6, 2020-03-19T21:18:32.169 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_epoch": 24, 2020-03-19T21:18:32.169 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_epoch_barrier": 16, 2020-03-19T21:18:32.170 INFO:teuthology.orchestra.run.smithi170.stdout: "uptime": 16.30126576 2020-03-19T21:18:32.170 INFO:teuthology.orchestra.run.smithi170.stdout:} 2020-03-19T21:18:32.176 INFO:tasks.cephfs.filesystem:_json_asok output: { "cluster_fsid": "12d0a33b-6b40-4398-90b4-6b5df1da388e", "whoami": 0, "id": 4202, "want_state": "up:active", "state": "up:active", "rank_uptime": 15.97159549, "mdsmap_epoch": 6, "osdmap_epoch": 24, "osdmap_epoch_barrier": 16, "uptime": 16.30126576 } 2020-03-19T21:18:32.177 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2020-03-19T21:18:32.177 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2020-03-19T21:18:32.177 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@smithi170.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2020-03-19T21:18:32.177 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-03-19T21:18:32.178 INFO:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest && exec mkdir -p /home/ubuntu/cephtest/mnt.1) 2020-03-19T21:18:32.196 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-03-19T21:18:32.196 INFO:teuthology.orchestra.run.smithi170:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-03-19T21:18:32.260 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-03-19T21:18:32.267 INFO:teuthology.orchestra.run.smithi170.stderr:mount: /sys/fs/fuse/connections: mount point does not exist. 2020-03-19T21:18:32.267 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-03-19T21:18:32.267 INFO:teuthology.orchestra.run.smithi170:> ls /sys/fs/fuse/connections 2020-03-19T21:18:32.314 DEBUG:teuthology.orchestra.run:got remote process result: 2 2020-03-19T21:18:32.319 INFO:teuthology.orchestra.run.smithi170.stderr:ls: cannot access '/sys/fs/fuse/connections': No such file or directory 2020-03-19T21:18:32.320 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2020-03-19T21:18:32.320 INFO:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2020-03-19T21:18:32.357 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-03-19T21:18:32.357 INFO:teuthology.orchestra.run.smithi170:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-03-19T21:18:32.386 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-03-19T21:18:32.392 INFO:teuthology.orchestra.run.smithi170.stderr:mount: /sys/fs/fuse/connections: mount point does not exist. 2020-03-19T21:18:32.392 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-03-19T21:18:32.392 INFO:teuthology.orchestra.run.smithi170:> ls /sys/fs/fuse/connections 2020-03-19T21:18:32.432 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi170.stderr:2020-03-19T21:18:32.431+0000 7f3b4440f1c0 -1 init, newargv = 0x55b82e886ee0 newargc=7 2020-03-19T21:18:32.433 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi170.stderr:ceph-fuse[23511]: starting ceph client 2020-03-19T21:18:32.443 DEBUG:teuthology.orchestra.run:got remote process result: 2 2020-03-19T21:18:32.446 INFO:teuthology.orchestra.run.smithi170.stderr:ls: cannot access '/sys/fs/fuse/connections': No such file or directory 2020-03-19T21:18:32.491 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi170.stderr:ceph-fuse[23511]: starting fuse 2020-03-19T21:18:33.547 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-03-19T21:18:33.548 INFO:teuthology.orchestra.run.smithi170:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-03-19T21:18:33.584 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-03-19T21:18:33.584 INFO:teuthology.orchestra.run.smithi170:> ls /sys/fs/fuse/connections 2020-03-19T21:18:33.640 INFO:teuthology.orchestra.run.smithi170.stdout:47 2020-03-19T21:18:33.640 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [47] 2020-03-19T21:18:33.640 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-03-19T21:18:33.640 INFO:teuthology.orchestra.run.smithi170:> sudo python3 -c ''"'"' 2020-03-19T21:18:33.640 INFO:teuthology.orchestra.run.smithi170:> import glob 2020-03-19T21:18:33.640 INFO:teuthology.orchestra.run.smithi170:> import re 2020-03-19T21:18:33.641 INFO:teuthology.orchestra.run.smithi170:> import os 2020-03-19T21:18:33.641 INFO:teuthology.orchestra.run.smithi170:> import subprocess 2020-03-19T21:18:33.641 INFO:teuthology.orchestra.run.smithi170:> 2020-03-19T21:18:33.641 INFO:teuthology.orchestra.run.smithi170:> def find_socket(client_name): 2020-03-19T21:18:33.641 INFO:teuthology.orchestra.run.smithi170:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2020-03-19T21:18:33.641 INFO:teuthology.orchestra.run.smithi170:> files = glob.glob(asok_path) 2020-03-19T21:18:33.641 INFO:teuthology.orchestra.run.smithi170:> 2020-03-19T21:18:33.642 INFO:teuthology.orchestra.run.smithi170:> # Given a non-glob path, it better be there 2020-03-19T21:18:33.642 INFO:teuthology.orchestra.run.smithi170:> if "*" not in asok_path: 2020-03-19T21:18:33.642 INFO:teuthology.orchestra.run.smithi170:> assert(len(files) == 1) 2020-03-19T21:18:33.642 INFO:teuthology.orchestra.run.smithi170:> return files[0] 2020-03-19T21:18:33.642 INFO:teuthology.orchestra.run.smithi170:> 2020-03-19T21:18:33.642 INFO:teuthology.orchestra.run.smithi170:> for f in files: 2020-03-19T21:18:33.642 INFO:teuthology.orchestra.run.smithi170:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2020-03-19T21:18:33.643 INFO:teuthology.orchestra.run.smithi170:> if os.path.exists("/proc/{0}".format(pid)): 2020-03-19T21:18:33.643 INFO:teuthology.orchestra.run.smithi170:> return f 2020-03-19T21:18:33.643 INFO:teuthology.orchestra.run.smithi170:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2020-03-19T21:18:33.643 INFO:teuthology.orchestra.run.smithi170:> 2020-03-19T21:18:33.643 INFO:teuthology.orchestra.run.smithi170:> print(find_socket("client.1")) 2020-03-19T21:18:33.644 INFO:teuthology.orchestra.run.smithi170:> '"'"'' 2020-03-19T21:18:33.731 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-03-19T21:18:33.745 INFO:teuthology.orchestra.run.smithi170.stderr: File "", line 1 2020-03-19T21:18:33.746 INFO:teuthology.orchestra.run.smithi170.stderr: ' 2020-03-19T21:18:33.746 INFO:teuthology.orchestra.run.smithi170.stderr: ^ 2020-03-19T21:18:33.746 INFO:teuthology.orchestra.run.smithi170.stderr:SyntaxError: EOL while scanning string literal 2020-03-19T21:18:33.746 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/run_tasks.py", line 89, in run_tasks manager.__enter__() File "/usr/lib/python2.7/contextlib.py", line 17, in __enter__ return self.gen.next() File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-sage-testing-2020-03-19-0855/qa/tasks/ceph_fuse.py", line 140, in task info["mount"].mount(mountpoint=mountpoint, mount_path=mount_path) File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-sage-testing-2020-03-19-0855/qa/tasks/cephfs/fuse_mount.py", line 33, in mount return self._mount(mount_path, mount_fs_name, mount_options) File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-sage-testing-2020-03-19-0855/qa/tasks/cephfs/fuse_mount.py", line 163, in _mount self.gather_mount_info() File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-sage-testing-2020-03-19-0855/qa/tasks/cephfs/fuse_mount.py", line 166, in gather_mount_info status = self.admin_socket(['status']) File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-sage-testing-2020-03-19-0855/qa/tasks/cephfs/fuse_mount.py", line 449, in admin_socket ], stdout=StringIO(), timeout=(15*60)) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/remote.py", line 198, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 433, in run r.wait() File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 158, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 180, in _raise_for_status node=self.hostname, label=self.label CommandFailedError: Command failed on smithi170 with status 1: 'sudo python3 -c \'\'"\'"\'\nimport glob\nimport re\nimport os\nimport subprocess\n\ndef find_socket(client_name):\n asok_path = "/var/run/ceph/ceph-client.1.*.asok"\n files = glob.glob(asok_path)\n\n # Given a non-glob path, it better be there\n if "*" not in asok_path:\n assert(len(files) == 1)\n return files[0]\n\n for f in files:\n pid = re.match(".*\\.(\\d+)\\.asok$", f).group(1)\n if os.path.exists("/proc/{0}".format(pid)):\n return f\n raise RuntimeError("Client socket {0} not found".format(client_name))\n\nprint(find_socket("client.1"))\n\'"\'"\'\'' 2020-03-19T21:18:33.750 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2020-03-19T21:18:33.763 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2020-03-19T21:18:33.792 INFO:tasks.ceph.ceph_manager.ceph:Canceling any pending splits or merges... 2020-03-19T21:18:33.792 INFO:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-03-19T21:18:33.903 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:33.902+0000 7ffaaf867700 1 Processor -- start 2020-03-19T21:18:33.903 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:33.902+0000 7ffaaf867700 1 -- start start 2020-03-19T21:18:33.908 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:33.902+0000 7ffaaf867700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7ffaa8125250 0x7ffaa8127640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T21:18:33.908 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:33.902+0000 7ffaaf867700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffaa808eee0 con 0x7ffaa81221d0 2020-03-19T21:18:33.908 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:33.902+0000 7ffaaf867700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffaa8091060 con 0x7ffaa8125250 2020-03-19T21:18:33.909 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:33.903+0000 7ffaace02700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7ffaa8125250 0x7ffaa8127640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:33.909 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:33.903+0000 7ffaad603700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7ffaa81221d0 0x7ffaa81225b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:54044/0 (socket says 172.21.15.170:54044) 2020-03-19T21:18:33.909 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:33.903+0000 7ffaad603700 1 -- 172.21.15.170:0/1009064592 learned_addr learned my addr 172.21.15.170:0/1009064592 (peer_addr_for_me v1:172.21.15.170:0/0) 2020-03-19T21:18:33.909 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:33.903+0000 7ffaace02700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7ffaa8125250 0x7ffaa8127640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53278/0 (socket says 172.21.15.170:53278) 2020-03-19T21:18:33.909 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:33.903+0000 7ffaace02700 1 -- 172.21.15.170:0/1009064592 >> v1:172.21.15.170:6789/0 conn(0x7ffaa81221d0 legacy=0x7ffaa81225b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:33.910 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:33.903+0000 7ffaace02700 1 -- 172.21.15.170:0/1009064592 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaa8121fd0 con 0x7ffaa8125250 2020-03-19T21:18:33.910 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:33.903+0000 7ffaace02700 1 --2- 172.21.15.170:0/1009064592 >> v2:172.21.15.170:3300/0 conn(0x7ffaa8125250 0x7ffaa8127640 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7ffaa0001450 tx=0x7ffa9c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:33.910 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:33.903+0000 7ffaace02700 1 --2- 172.21.15.170:0/1009064592 >> v2:172.21.15.170:3300/0 conn(0x7ffaa8125250 0x7ffaa8127640 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7ffaa0001450 tx=0x7ffa9c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:33.910 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:33.903+0000 7ffaace02700 1 --2- 172.21.15.170:0/1009064592 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffaa8125250 0x7ffaa8127640 secure :-1 s=READY pgs=259 cs=0 l=1 rx=0x7ffaa0001450 tx=0x7ffa9c007a40).ready entity=mon.0 client_cookie=559aad96e58f683 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:33.910 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:33.903+0000 7ffaace02700 1 --2- 172.21.15.170:0/1009064592 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffaa8125250 0x7ffaa8127640 secure :-1 s=THROTTLE_DONE pgs=259 cs=0 l=1 rx=0x7ffaa0001450 tx=0x7ffa9c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:33.910 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:33.903+0000 7ffa9bfff700 1 -- 172.21.15.170:0/1009064592 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7ffa9c008bb0 con 0x7ffaa8125250 2020-03-19T21:18:33.910 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:33.903+0000 7ffaace02700 1 --2- 172.21.15.170:0/1009064592 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffaa8125250 0x7ffaa8127640 secure :-1 s=THROTTLE_DONE pgs=259 cs=0 l=1 rx=0x7ffaa0001450 tx=0x7ffa9c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:33.911 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:33.903+0000 7ffaace02700 1 --2- 172.21.15.170:0/1009064592 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffaa8125250 0x7ffaa8127640 secure :-1 s=THROTTLE_DONE pgs=259 cs=0 l=1 rx=0x7ffaa0001450 tx=0x7ffa9c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:33.911 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:33.903+0000 7ffa9bfff700 1 -- 172.21.15.170:0/1009064592 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa9c008d10 con 0x7ffaa8125250 2020-03-19T21:18:33.911 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:33.903+0000 7ffa9bfff700 1 -- 172.21.15.170:0/1009064592 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7ffa9c011890 con 0x7ffaa8125250 2020-03-19T21:18:33.911 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:33.903+0000 7ffaaf867700 1 -- 172.21.15.170:0/1009064592 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffaa8125250 msgr2=0x7ffaa8127640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:33.911 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:33.903+0000 7ffaaf867700 1 --2- 172.21.15.170:0/1009064592 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffaa8125250 0x7ffaa8127640 secure :-1 s=READY pgs=259 cs=0 l=1 rx=0x7ffaa0001450 tx=0x7ffa9c007a40).stop 2020-03-19T21:18:33.911 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:33.904+0000 7ffaaf867700 1 -- 172.21.15.170:0/1009064592 shutdown_connections 2020-03-19T21:18:33.912 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:33.904+0000 7ffaaf867700 1 --2- 172.21.15.170:0/1009064592 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffaa8125250 0x7ffaa8127640 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:33.912 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:33.904+0000 7ffaaf867700 1 -- 172.21.15.170:0/1009064592 >> 172.21.15.170:0/1009064592 conn(0x7ffaa8091540 msgr2=0x7ffaa8091950 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:33.912 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:33.904+0000 7ffaaf867700 1 -- 172.21.15.170:0/1009064592 shutdown_connections 2020-03-19T21:18:33.912 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:33.904+0000 7ffaaf867700 1 -- 172.21.15.170:0/1009064592 wait complete. 2020-03-19T21:18:33.914 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:33.905+0000 7ffaaf867700 1 Processor -- start 2020-03-19T21:18:33.914 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:33.905+0000 7ffaaf867700 1 -- start start 2020-03-19T21:18:33.914 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:33.905+0000 7ffaaf867700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7ffaa8125250 0x7ffaa8066c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:33.914 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:33.905+0000 7ffaaf867700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffaa8064310 con 0x7ffaa81221d0 2020-03-19T21:18:33.915 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:33.905+0000 7ffaaf867700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffaa8063210 con 0x7ffaa8125250 2020-03-19T21:18:33.916 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:33.905+0000 7ffaace02700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7ffaa8125250 0x7ffaa8066c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:33.916 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:33.905+0000 7ffaad603700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7ffaa81221d0 0x7ffaa8066530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:54046/0 (socket says 172.21.15.170:54046) 2020-03-19T21:18:33.916 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:33.905+0000 7ffaad603700 1 -- 172.21.15.170:0/1235415640 learned_addr learned my addr 172.21.15.170:0/1235415640 (peer_addr_for_me v1:172.21.15.170:0/0) 2020-03-19T21:18:33.916 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:33.905+0000 7ffaace02700 1 -- 172.21.15.170:0/1235415640 >> v1:172.21.15.170:6789/0 conn(0x7ffaa81221d0 legacy=0x7ffaa8066530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:33.917 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:33.905+0000 7ffa9a7fc700 1 -- 172.21.15.170:0/1235415640 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2951563626 0 0) 0x7ffaa8064310 con 0x7ffaa81221d0 2020-03-19T21:18:33.917 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:33.905+0000 7ffaace02700 1 -- 172.21.15.170:0/1235415640 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaa8064000 con 0x7ffaa8125250 2020-03-19T21:18:33.917 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:33.905+0000 7ffaace02700 1 --2- 172.21.15.170:0/1235415640 >> v2:172.21.15.170:3300/0 conn(0x7ffaa8125250 0x7ffaa8066c40 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7ffaa0001450 tx=0x7ffa9c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:33.917 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:33.905+0000 7ffaace02700 1 --2- 172.21.15.170:0/1235415640 >> v2:172.21.15.170:3300/0 conn(0x7ffaa8125250 0x7ffaa8066c40 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7ffaa0001450 tx=0x7ffa9c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:33.917 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:33.905+0000 7ffaace02700 1 --2- 172.21.15.170:0/1235415640 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffaa8125250 0x7ffaa8066c40 secure :-1 s=READY pgs=261 cs=0 l=1 rx=0x7ffaa0001450 tx=0x7ffa9c007a40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:33.917 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:33.906+0000 7ffaace02700 1 --2- 172.21.15.170:0/1235415640 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffaa8125250 0x7ffaa8066c40 secure :-1 s=THROTTLE_DONE pgs=261 cs=0 l=1 rx=0x7ffaa0001450 tx=0x7ffa9c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:33.917 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:33.906+0000 7ffa9a7fc700 1 -- 172.21.15.170:0/1235415640 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7ffa9c00f5e0 con 0x7ffaa8125250 2020-03-19T21:18:33.918 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:33.906+0000 7ffaace02700 1 --2- 172.21.15.170:0/1235415640 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffaa8125250 0x7ffaa8066c40 secure :-1 s=THROTTLE_DONE pgs=261 cs=0 l=1 rx=0x7ffaa0001450 tx=0x7ffa9c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:33.918 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:33.906+0000 7ffa9a7fc700 1 -- 172.21.15.170:0/1235415640 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa9c00f740 con 0x7ffaa8125250 2020-03-19T21:18:33.918 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:33.906+0000 7ffaace02700 1 --2- 172.21.15.170:0/1235415640 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffaa8125250 0x7ffaa8066c40 secure :-1 s=THROTTLE_DONE pgs=261 cs=0 l=1 rx=0x7ffaa0001450 tx=0x7ffa9c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:33.918 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:33.906+0000 7ffaaf867700 1 -- 172.21.15.170:0/1235415640 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffaa8063b70 con 0x7ffaa8125250 2020-03-19T21:18:33.918 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:33.906+0000 7ffa9a7fc700 1 -- 172.21.15.170:0/1235415640 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7ffa9c007160 con 0x7ffaa8125250 2020-03-19T21:18:33.918 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:33.906+0000 7ffaaf867700 1 -- 172.21.15.170:0/1235415640 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffaa8062c50 con 0x7ffaa8125250 2020-03-19T21:18:33.918 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:33.906+0000 7ffaace02700 1 --2- 172.21.15.170:0/1235415640 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffaa8125250 0x7ffaa8066c40 secure :-1 s=THROTTLE_DONE pgs=261 cs=0 l=1 rx=0x7ffaa0001450 tx=0x7ffa9c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:33.919 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:33.906+0000 7ffa9a7fc700 1 -- 172.21.15.170:0/1235415640 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 4) v1 ==== 29603+0+0 (secure 0 0 0) 0x7ffa9c007360 con 0x7ffaa8125250 2020-03-19T21:18:33.919 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:33.906+0000 7ffaace02700 1 --2- 172.21.15.170:0/1235415640 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffaa8125250 0x7ffaa8066c40 secure :-1 s=THROTTLE_DONE pgs=261 cs=0 l=1 rx=0x7ffaa0001450 tx=0x7ffa9c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:33.919 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:33.906+0000 7ffa9a7fc700 1 --2- 172.21.15.170:0/1235415640 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7ffa88028200 0x7ffa8802a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:33.919 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:33.906+0000 7ffa9a7fc700 1 -- 172.21.15.170:0/1235415640 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 3786+0+0 (secure 0 0 0) 0x7ffa9c035030 con 0x7ffaa8125250 2020-03-19T21:18:33.919 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:33.906+0000 7ffaad603700 1 --2- 172.21.15.170:0/1235415640 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7ffa88028200 0x7ffa8802a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:33.920 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:33.907+0000 7ffaad603700 1 --2- 172.21.15.170:0/1235415640 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7ffa88028200 0x7ffa8802a6c0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7ffaa0001380 tx=0x7ffaa0001b50).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:33.920 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:33.907+0000 7ffaad603700 1 --2- 172.21.15.170:0/1235415640 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7ffa88028200 0x7ffa8802a6c0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7ffaa0001380 tx=0x7ffaa0001b50).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:33.920 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:33.907+0000 7ffaad603700 1 --2- 172.21.15.170:0/1235415640 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7ffa88028200 0x7ffa8802a6c0 secure :-1 s=READY pgs=64 cs=0 l=1 rx=0x7ffaa0001380 tx=0x7ffaa0001b50).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:33.920 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:33.907+0000 7ffaaf867700 1 -- 172.21.15.170:0/1235415640 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffaa811f540 con 0x7ffaa8125250 2020-03-19T21:18:33.920 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:33.910+0000 7ffaace02700 1 --2- 172.21.15.170:0/1235415640 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffaa8125250 0x7ffaa8066c40 secure :-1 s=THROTTLE_DONE pgs=261 cs=0 l=1 rx=0x7ffaa0001450 tx=0x7ffa9c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:33.921 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:33.910+0000 7ffa9a7fc700 1 -- 172.21.15.170:0/1235415640 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+126772 (secure 0 0 0) 0x7ffa9c006870 con 0x7ffaa8125250 2020-03-19T21:18:34.050 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.049+0000 7ffaaf867700 1 -- 172.21.15.170:0/1235415640 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffaa8056db0 con 0x7ffaa8125250 2020-03-19T21:18:34.051 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.050+0000 7ffaace02700 1 --2- 172.21.15.170:0/1235415640 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffaa8125250 0x7ffaa8066c40 secure :-1 s=THROTTLE_DONE pgs=261 cs=0 l=1 rx=0x7ffaa0001450 tx=0x7ffa9c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:34.051 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.050+0000 7ffa9a7fc700 1 -- 172.21.15.170:0/1235415640 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+10376 (secure 0 0 0) 0x7ffa9c006870 con 0x7ffaa8125250 2020-03-19T21:18:34.052 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:18:34.052 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":24,"fsid":"12d0a33b-6b40-4398-90b4-6b5df1da388e","created":"2020-03-19T21:17:59.135747+0000","modified":"2020-03-19T21:18:30.247068+0000","last_up_change":"2020-03-19T21:18:05.158515+0000","last_in_change":"2020-03-19T21:18:05.158515+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":3,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2020-03-19T21:18:08.391604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2020-03-19T21:18:11.286536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2020-03-19T21:18:12.597110+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"20","last_force_op_resend_prenautilus":"20","last_force_op_resend_preluminous":"20","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":18,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[4],"tier_of":-1,"read_tier":4,"write_tier":4,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":4,"pool_name":"data_cache","create_time":"2020-03-19T21:18:22.941000+0000","flags":9,"flags_names":"hashpspool,incomplete_clones","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":4,"pg_placement_num":4,"pg_placement_num_target":4,"pg_num_target":4,"pg_num_pending":4,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"20","last_force_op_resend_prenautilus":"20","last_force_op_resend_preluminous":"20","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":18,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":3,"read_tier":-1,"write_tier":-1,"cache_mode":"writeback","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"bloom","false_positive_probability":0.050000000000000003,"target_size":0,"seed":0},"hit_set_period":3600,"hit_set_count":8,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[1000000,1000000,1000000,1000000,1000000,1000000,1000000,1000000],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{}}],"osds":[{"osd":0,"uuid":"32ca3517-1077-4102-9aa4-edfcebe8e3d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":20680},{"type":"v1","addr":"172.21.15.170:6801","nonce":20680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":20680},{"type":"v1","addr":"172.21.15.170:6803","nonce":20680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":20680},{"type":"v1","addr":"172.21.15.170:6807","nonce":20680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":20680},{"type":"v1","addr":"172.21.15.170:6805","nonce":20680}]},"public_addr":"172.21.15.170:6801/20680","cluster_addr":"172.21.15.170:6803/20680","heartbeat_back_addr":"172.21.15.170:6807/20680","heartbeat_front_addr":"172.21.15.170:6805/20680","state":["exists","up"]},{"osd":1,"uuid":"470df64b-60bc-4f5b-9b53-64495cd7c5e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":20681},{"type":"v1","addr":"172.21.15.170:6809","nonce":20681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":20681},{"type":"v1","addr":"172.21.15.170:6811","nonce":20681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":20681},{"type":"v1","addr":"172.21.15.170:6815","nonce":20681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":20681},{"type":"v1","addr":"172.21.15.170:6813","nonce":20681}]},"public_addr":"172.21.15.170:6809/20681","cluster_addr":"172.21.15.170:6811/20681","heartbeat_back_addr":"172.21.15.170:6815/20681","heartbeat_front_addr":"172.21.15.170:6813/20681","state":["exists","up"]},{"osd":2,"uuid":"ccfb9d82-b759-4ff4-8ddf-4f1d741bac5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":20682},{"type":"v1","addr":"172.21.15.170:6817","nonce":20682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":20682},{"type":"v1","addr":"172.21.15.170:6819","nonce":20682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":20682},{"type":"v1","addr":"172.21.15.170:6823","nonce":20682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":20682},{"type":"v1","addr":"172.21.15.170:6821","nonce":20682}]},"public_addr":"172.21.15.170:6817/20682","cluster_addr":"172.21.15.170:6819/20682","heartbeat_back_addr":"172.21.15.170:6823/20682","heartbeat_front_addr":"172.21.15.170:6821/20682","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2020-03-19T21:18:03.177293+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2020-03-19T21:18:03.263179+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2020-03-19T21:18:03.183223+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-03-19T21:18:34.053 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.051+0000 7ffaaf867700 1 -- 172.21.15.170:0/1235415640 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7ffa88028200 msgr2=0x7ffa8802a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:34.053 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.051+0000 7ffaaf867700 1 --2- 172.21.15.170:0/1235415640 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7ffa88028200 0x7ffa8802a6c0 secure :-1 s=READY pgs=64 cs=0 l=1 rx=0x7ffaa0001380 tx=0x7ffaa0001b50).stop 2020-03-19T21:18:34.054 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.051+0000 7ffaaf867700 1 -- 172.21.15.170:0/1235415640 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffaa8125250 msgr2=0x7ffaa8066c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:34.054 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.051+0000 7ffaaf867700 1 --2- 172.21.15.170:0/1235415640 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffaa8125250 0x7ffaa8066c40 secure :-1 s=READY pgs=261 cs=0 l=1 rx=0x7ffaa0001450 tx=0x7ffa9c007a40).stop 2020-03-19T21:18:34.054 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.051+0000 7ffaaf867700 1 -- 172.21.15.170:0/1235415640 shutdown_connections 2020-03-19T21:18:34.054 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.051+0000 7ffaaf867700 1 --2- 172.21.15.170:0/1235415640 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7ffa88028200 0x7ffa8802a6c0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:34.054 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.051+0000 7ffaaf867700 1 --2- 172.21.15.170:0/1235415640 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffaa8125250 0x7ffaa8066c40 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:34.054 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.052+0000 7ffaaf867700 1 -- 172.21.15.170:0/1235415640 >> 172.21.15.170:0/1235415640 conn(0x7ffaa8091540 msgr2=0x7ffaa808ced0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:34.054 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.052+0000 7ffaaf867700 1 -- 172.21.15.170:0/1235415640 shutdown_connections 2020-03-19T21:18:34.055 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.052+0000 7ffaaf867700 1 -- 172.21.15.170:0/1235415640 wait complete. 2020-03-19T21:18:34.063 INFO:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early pg dump --format=json 2020-03-19T21:18:34.168 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.167+0000 7fcc66c78700 1 Processor -- start 2020-03-19T21:18:34.168 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.167+0000 7fcc66c78700 1 -- start start 2020-03-19T21:18:34.172 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.167+0000 7fcc66c78700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fcc60125290 0x7fcc60127680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T21:18:34.172 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.167+0000 7fcc66c78700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc6008e840 con 0x7fcc601223d0 2020-03-19T21:18:34.172 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.167+0000 7fcc66c78700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc60091060 con 0x7fcc60125290 2020-03-19T21:18:34.172 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.168+0000 7fcc5ffff700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fcc60125290 0x7fcc60127680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:34.173 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.168+0000 7fcc64a14700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fcc601223d0 0x7fcc601227b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:54052/0 (socket says 172.21.15.170:54052) 2020-03-19T21:18:34.173 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.168+0000 7fcc64a14700 1 -- 172.21.15.170:0/555707762 learned_addr learned my addr 172.21.15.170:0/555707762 (peer_addr_for_me v1:172.21.15.170:0/0) 2020-03-19T21:18:34.173 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.168+0000 7fcc5ffff700 1 -- 172.21.15.170:0/555707762 >> v1:172.21.15.170:6789/0 conn(0x7fcc601223d0 legacy=0x7fcc601227b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:34.173 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.168+0000 7fcc5f7fe700 1 -- 172.21.15.170:0/555707762 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2309760045 0 0) 0x7fcc6008e840 con 0x7fcc601223d0 2020-03-19T21:18:34.173 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.168+0000 7fcc5ffff700 1 -- 172.21.15.170:0/555707762 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc60121f30 con 0x7fcc60125290 2020-03-19T21:18:34.173 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.168+0000 7fcc5ffff700 1 --2- 172.21.15.170:0/555707762 >> v2:172.21.15.170:3300/0 conn(0x7fcc60125290 0x7fcc60127680 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7fcc54002f30 tx=0x7fcc54007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:34.173 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.168+0000 7fcc5ffff700 1 --2- 172.21.15.170:0/555707762 >> v2:172.21.15.170:3300/0 conn(0x7fcc60125290 0x7fcc60127680 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7fcc54002f30 tx=0x7fcc54007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:34.174 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.168+0000 7fcc5ffff700 1 --2- 172.21.15.170:0/555707762 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcc60125290 0x7fcc60127680 secure :-1 s=READY pgs=263 cs=0 l=1 rx=0x7fcc54002f30 tx=0x7fcc54007a40).ready entity=mon.0 client_cookie=960a76bc39cb2d70 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:34.174 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.168+0000 7fcc5ffff700 1 --2- 172.21.15.170:0/555707762 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcc60125290 0x7fcc60127680 secure :-1 s=THROTTLE_DONE pgs=263 cs=0 l=1 rx=0x7fcc54002f30 tx=0x7fcc54007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:34.174 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.168+0000 7fcc5f7fe700 1 -- 172.21.15.170:0/555707762 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fcc54008e80 con 0x7fcc60125290 2020-03-19T21:18:34.174 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.168+0000 7fcc5ffff700 1 --2- 172.21.15.170:0/555707762 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcc60125290 0x7fcc60127680 secure :-1 s=THROTTLE_DONE pgs=263 cs=0 l=1 rx=0x7fcc54002f30 tx=0x7fcc54007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:34.174 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.168+0000 7fcc5ffff700 1 --2- 172.21.15.170:0/555707762 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcc60125290 0x7fcc60127680 secure :-1 s=THROTTLE_DONE pgs=263 cs=0 l=1 rx=0x7fcc54002f30 tx=0x7fcc54007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:34.174 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.168+0000 7fcc5f7fe700 1 -- 172.21.15.170:0/555707762 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc540113f0 con 0x7fcc60125290 2020-03-19T21:18:34.174 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.168+0000 7fcc5f7fe700 1 -- 172.21.15.170:0/555707762 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fcc540116d0 con 0x7fcc60125290 2020-03-19T21:18:34.175 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.168+0000 7fcc66c78700 1 -- 172.21.15.170:0/555707762 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcc60125290 msgr2=0x7fcc60127680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:34.175 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.168+0000 7fcc66c78700 1 --2- 172.21.15.170:0/555707762 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcc60125290 0x7fcc60127680 secure :-1 s=READY pgs=263 cs=0 l=1 rx=0x7fcc54002f30 tx=0x7fcc54007a40).stop 2020-03-19T21:18:34.175 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.169+0000 7fcc66c78700 1 -- 172.21.15.170:0/555707762 shutdown_connections 2020-03-19T21:18:34.175 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.169+0000 7fcc66c78700 1 --2- 172.21.15.170:0/555707762 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcc60125290 0x7fcc60127680 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:34.175 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.169+0000 7fcc66c78700 1 -- 172.21.15.170:0/555707762 >> 172.21.15.170:0/555707762 conn(0x7fcc60091540 msgr2=0x7fcc60091950 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:34.175 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.169+0000 7fcc66c78700 1 -- 172.21.15.170:0/555707762 shutdown_connections 2020-03-19T21:18:34.175 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.169+0000 7fcc66c78700 1 -- 172.21.15.170:0/555707762 wait complete. 2020-03-19T21:18:34.176 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.169+0000 7fcc66c78700 1 Processor -- start 2020-03-19T21:18:34.176 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.169+0000 7fcc66c78700 1 -- start start 2020-03-19T21:18:34.176 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.170+0000 7fcc66c78700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fcc60125290 0x7fcc60066f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:34.176 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.170+0000 7fcc66c78700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc60061390 con 0x7fcc601223d0 2020-03-19T21:18:34.176 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.170+0000 7fcc66c78700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc60067360 con 0x7fcc60125290 2020-03-19T21:18:34.176 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.170+0000 7fcc5ffff700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fcc60125290 0x7fcc60066f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:34.176 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.170+0000 7fcc64a14700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fcc601223d0 0x7fcc600667f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:54056/0 (socket says 172.21.15.170:54056) 2020-03-19T21:18:34.176 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.170+0000 7fcc64a14700 1 -- 172.21.15.170:0/2616088228 learned_addr learned my addr 172.21.15.170:0/2616088228 (peer_addr_for_me v1:172.21.15.170:0/0) 2020-03-19T21:18:34.177 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.170+0000 7fcc5ffff700 1 -- 172.21.15.170:0/2616088228 >> v1:172.21.15.170:6789/0 conn(0x7fcc601223d0 legacy=0x7fcc600667f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:34.177 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.170+0000 7fcc5dffb700 1 -- 172.21.15.170:0/2616088228 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 644859186 0 0) 0x7fcc60061390 con 0x7fcc601223d0 2020-03-19T21:18:34.177 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.170+0000 7fcc5ffff700 1 -- 172.21.15.170:0/2616088228 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc60062920 con 0x7fcc60125290 2020-03-19T21:18:34.177 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.170+0000 7fcc5ffff700 1 --2- 172.21.15.170:0/2616088228 >> v2:172.21.15.170:3300/0 conn(0x7fcc60125290 0x7fcc60066f00 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7fcc54002f30 tx=0x7fcc54007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:34.177 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.170+0000 7fcc5ffff700 1 --2- 172.21.15.170:0/2616088228 >> v2:172.21.15.170:3300/0 conn(0x7fcc60125290 0x7fcc60066f00 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7fcc54002f30 tx=0x7fcc54007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:34.177 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.170+0000 7fcc5ffff700 1 --2- 172.21.15.170:0/2616088228 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcc60125290 0x7fcc60066f00 secure :-1 s=READY pgs=265 cs=0 l=1 rx=0x7fcc54002f30 tx=0x7fcc54007a40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:34.177 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.170+0000 7fcc5ffff700 1 --2- 172.21.15.170:0/2616088228 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcc60125290 0x7fcc60066f00 secure :-1 s=THROTTLE_DONE pgs=265 cs=0 l=1 rx=0x7fcc54002f30 tx=0x7fcc54007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:34.177 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.170+0000 7fcc5dffb700 1 -- 172.21.15.170:0/2616088228 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fcc540116c0 con 0x7fcc60125290 2020-03-19T21:18:34.178 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.170+0000 7fcc5ffff700 1 --2- 172.21.15.170:0/2616088228 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcc60125290 0x7fcc60066f00 secure :-1 s=THROTTLE_DONE pgs=265 cs=0 l=1 rx=0x7fcc54002f30 tx=0x7fcc54007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:34.178 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.171+0000 7fcc5ffff700 1 --2- 172.21.15.170:0/2616088228 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcc60125290 0x7fcc60066f00 secure :-1 s=THROTTLE_DONE pgs=265 cs=0 l=1 rx=0x7fcc54002f30 tx=0x7fcc54007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:34.178 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.171+0000 7fcc5dffb700 1 -- 172.21.15.170:0/2616088228 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc5400f830 con 0x7fcc60125290 2020-03-19T21:18:34.178 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.171+0000 7fcc66c78700 1 -- 172.21.15.170:0/2616088228 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc60064290 con 0x7fcc60125290 2020-03-19T21:18:34.178 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.171+0000 7fcc5dffb700 1 -- 172.21.15.170:0/2616088228 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fcc54007280 con 0x7fcc60125290 2020-03-19T21:18:34.178 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.171+0000 7fcc66c78700 1 -- 172.21.15.170:0/2616088228 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc60063340 con 0x7fcc60125290 2020-03-19T21:18:34.179 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.171+0000 7fcc5ffff700 1 --2- 172.21.15.170:0/2616088228 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcc60125290 0x7fcc60066f00 secure :-1 s=THROTTLE_DONE pgs=265 cs=0 l=1 rx=0x7fcc54002f30 tx=0x7fcc54007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:34.180 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.171+0000 7fcc5dffb700 1 -- 172.21.15.170:0/2616088228 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 4) v1 ==== 29603+0+0 (secure 0 0 0) 0x7fcc540074b0 con 0x7fcc60125290 2020-03-19T21:18:34.180 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.171+0000 7fcc5ffff700 1 --2- 172.21.15.170:0/2616088228 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcc60125290 0x7fcc60066f00 secure :-1 s=THROTTLE_DONE pgs=265 cs=0 l=1 rx=0x7fcc54002f30 tx=0x7fcc54007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:34.180 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.171+0000 7fcc5dffb700 1 --2- 172.21.15.170:0/2616088228 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7fcc40028250 0x7fcc4002a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:34.180 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.171+0000 7fcc5dffb700 1 -- 172.21.15.170:0/2616088228 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 3786+0+0 (secure 0 0 0) 0x7fcc540348e0 con 0x7fcc60125290 2020-03-19T21:18:34.180 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.172+0000 7fcc64a14700 1 --2- 172.21.15.170:0/2616088228 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7fcc40028250 0x7fcc4002a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:34.181 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.172+0000 7fcc66c78700 1 -- 172.21.15.170:0/2616088228 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc60062b10 con 0x7fcc60125290 2020-03-19T21:18:34.181 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.172+0000 7fcc64a14700 1 --2- 172.21.15.170:0/2616088228 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7fcc40028250 0x7fcc4002a710 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7fcc580015e0 tx=0x7fcc58001aa0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:34.181 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.172+0000 7fcc64a14700 1 --2- 172.21.15.170:0/2616088228 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7fcc40028250 0x7fcc4002a710 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7fcc580015e0 tx=0x7fcc58001aa0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:34.181 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.172+0000 7fcc64a14700 1 --2- 172.21.15.170:0/2616088228 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7fcc40028250 0x7fcc4002a710 secure :-1 s=READY pgs=65 cs=0 l=1 rx=0x7fcc580015e0 tx=0x7fcc58001aa0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:34.181 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.175+0000 7fcc5ffff700 1 --2- 172.21.15.170:0/2616088228 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcc60125290 0x7fcc60066f00 secure :-1 s=THROTTLE_DONE pgs=265 cs=0 l=1 rx=0x7fcc54002f30 tx=0x7fcc54007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:34.181 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.175+0000 7fcc5dffb700 1 -- 172.21.15.170:0/2616088228 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+126772 (secure 0 0 0) 0x7fcc54009c40 con 0x7fcc60125290 2020-03-19T21:18:34.312 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.311+0000 7fcc66c78700 1 -- 172.21.15.170:0/2616088228 --> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcc600635c0 con 0x7fcc40028250 2020-03-19T21:18:34.315 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.313+0000 7fcc64a14700 1 --2- 172.21.15.170:0/2616088228 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7fcc40028250 0x7fcc4002a710 secure :-1 s=THROTTLE_DONE pgs=65 cs=0 l=1 rx=0x7fcc580015e0 tx=0x7fcc58001aa0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:34.315 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.313+0000 7fcc5dffb700 1 -- 172.21.15.170:0/2616088228 <== mgr.4102 v2:172.21.15.170:6824/20154 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+74048 (secure 0 0 0) 0x7fcc600635c0 con 0x7fcc40028250 2020-03-19T21:18:34.315 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:18:34.317 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.314+0000 7fcc66c78700 1 -- 172.21.15.170:0/2616088228 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7fcc40028250 msgr2=0x7fcc4002a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:34.317 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.315+0000 7fcc66c78700 1 --2- 172.21.15.170:0/2616088228 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7fcc40028250 0x7fcc4002a710 secure :-1 s=READY pgs=65 cs=0 l=1 rx=0x7fcc580015e0 tx=0x7fcc58001aa0).stop 2020-03-19T21:18:34.317 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.315+0000 7fcc66c78700 1 -- 172.21.15.170:0/2616088228 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcc60125290 msgr2=0x7fcc60066f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:34.317 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.315+0000 7fcc66c78700 1 --2- 172.21.15.170:0/2616088228 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcc60125290 0x7fcc60066f00 secure :-1 s=READY pgs=265 cs=0 l=1 rx=0x7fcc54002f30 tx=0x7fcc54007a40).stop 2020-03-19T21:18:34.317 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.315+0000 7fcc66c78700 1 -- 172.21.15.170:0/2616088228 shutdown_connections 2020-03-19T21:18:34.318 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.315+0000 7fcc66c78700 1 --2- 172.21.15.170:0/2616088228 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7fcc40028250 0x7fcc4002a710 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:34.318 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.315+0000 7fcc66c78700 1 --2- 172.21.15.170:0/2616088228 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcc60125290 0x7fcc60066f00 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:34.318 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.315+0000 7fcc66c78700 1 -- 172.21.15.170:0/2616088228 >> 172.21.15.170:0/2616088228 conn(0x7fcc60091540 msgr2=0x7fcc60093310 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:34.318 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.315+0000 7fcc66c78700 1 -- 172.21.15.170:0/2616088228 shutdown_connections 2020-03-19T21:18:34.318 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.315+0000 7fcc66c78700 1 -- 172.21.15.170:0/2616088228 wait complete. 2020-03-19T21:18:34.318 INFO:teuthology.orchestra.run.smithi170.stderr:dumped all 2020-03-19T21:18:34.326 INFO:teuthology.orchestra.run.smithi170.stdout:{"pg_ready":true,"pg_map":{"version":36,"stamp":"2020-03-19T21:18:32.512057+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2286,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":314572800,"kb_used":3146384,"kb_used_data":632,"kb_used_omap":0,"kb_used_meta":3145728,"kb_avail":311426416,"statfs":{"total":322122547200,"available":318900649984,"internally_reserved":3221225472,"allocated":647168,"data_stored":220501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":3221225472},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.276123"},"pg_stats":[{"pgid":"4.0","version":"0'0","reported_seq":"12","reported_epoch":"22","state":"active+clean","last_fresh":"2020-03-19T21:18:29.006684+0000","last_change":"2020-03-19T21:18:24.224546+0000","last_active":"2020-03-19T21:18:29.006684+0000","last_peered":"2020-03-19T21:18:29.006684+0000","last_clean":"2020-03-19T21:18:29.006684+0000","last_became_active":"2020-03-19T21:18:24.224136+0000","last_became_peered":"2020-03-19T21:18:24.224136+0000","last_unstale":"2020-03-19T21:18:29.006684+0000","last_undegraded":"2020-03-19T21:18:29.006684+0000","last_fullsized":"2020-03-19T21:18:29.006684+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:23.204329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:23.204329+0000","last_clean_scrub_stamp":"2020-03-19T21:18:23.204329+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":"16","reported_epoch":"23","state":"active+clean","last_fresh":"2020-03-19T21:18:30.007436+0000","last_change":"2020-03-19T21:18:13.525398+0000","last_active":"2020-03-19T21:18:30.007436+0000","last_peered":"2020-03-19T21:18:30.007436+0000","last_clean":"2020-03-19T21:18:30.007436+0000","last_became_active":"2020-03-19T21:18:13.525041+0000","last_became_peered":"2020-03-19T21:18:13.525041+0000","last_unstale":"2020-03-19T21:18:30.007436+0000","last_undegraded":"2020-03-19T21:18:30.007436+0000","last_fullsized":"2020-03-19T21:18:30.007436+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:13.185694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:13.185694+0000","last_clean_scrub_stamp":"2020-03-19T21:18:13.185694+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":"20","reported_epoch":"23","state":"active+clean","last_fresh":"2020-03-19T21:18:30.007878+0000","last_change":"2020-03-19T21:18:10.186965+0000","last_active":"2020-03-19T21:18:30.007878+0000","last_peered":"2020-03-19T21:18:30.007878+0000","last_clean":"2020-03-19T21:18:30.007878+0000","last_became_active":"2020-03-19T21:18:10.186584+0000","last_became_peered":"2020-03-19T21:18:10.186584+0000","last_unstale":"2020-03-19T21:18:30.007878+0000","last_undegraded":"2020-03-19T21:18:30.007878+0000","last_fullsized":"2020-03-19T21:18:30.007878+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:09.169195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:09.169195+0000","last_clean_scrub_stamp":"2020-03-19T21:18:09.169195+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"16'4","reported_seq":"22","reported_epoch":"23","state":"active+clean","last_fresh":"2020-03-19T21:18:29.245838+0000","last_change":"2020-03-19T21:18:13.197989+0000","last_active":"2020-03-19T21:18:29.245838+0000","last_peered":"2020-03-19T21:18:29.245838+0000","last_clean":"2020-03-19T21:18:29.245838+0000","last_became_active":"2020-03-19T21:18:13.197671+0000","last_became_peered":"2020-03-19T21:18:13.197671+0000","last_unstale":"2020-03-19T21:18:29.245838+0000","last_undegraded":"2020-03-19T21:18:29.245838+0000","last_fullsized":"2020-03-19T21:18:29.245838+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:12.182524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:12.182524+0000","last_clean_scrub_stamp":"2020-03-19T21:18:12.182524+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":"13","reported_epoch":"23","state":"active+clean","last_fresh":"2020-03-19T21:18:30.007827+0000","last_change":"2020-03-19T21:18:24.224285+0000","last_active":"2020-03-19T21:18:30.007827+0000","last_peered":"2020-03-19T21:18:30.007827+0000","last_clean":"2020-03-19T21:18:30.007827+0000","last_became_active":"2020-03-19T21:18:24.224025+0000","last_became_peered":"2020-03-19T21:18:24.224025+0000","last_unstale":"2020-03-19T21:18:30.007827+0000","last_undegraded":"2020-03-19T21:18:30.007827+0000","last_fullsized":"2020-03-19T21:18:30.007827+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:23.204329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:23.204329+0000","last_clean_scrub_stamp":"2020-03-19T21:18:23.204329+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":"15","reported_epoch":"22","state":"active+clean","last_fresh":"2020-03-19T21:18:29.006832+0000","last_change":"2020-03-19T21:18:13.526027+0000","last_active":"2020-03-19T21:18:29.006832+0000","last_peered":"2020-03-19T21:18:29.006832+0000","last_clean":"2020-03-19T21:18:29.006832+0000","last_became_active":"2020-03-19T21:18:13.524563+0000","last_became_peered":"2020-03-19T21:18:13.524563+0000","last_unstale":"2020-03-19T21:18:29.006832+0000","last_undegraded":"2020-03-19T21:18:29.006832+0000","last_fullsized":"2020-03-19T21:18:29.006832+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:13.185694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:13.185694+0000","last_clean_scrub_stamp":"2020-03-19T21:18:13.185694+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":"20","reported_epoch":"23","state":"active+clean","last_fresh":"2020-03-19T21:18:29.245592+0000","last_change":"2020-03-19T21:18:10.184204+0000","last_active":"2020-03-19T21:18:29.245592+0000","last_peered":"2020-03-19T21:18:29.245592+0000","last_clean":"2020-03-19T21:18:29.245592+0000","last_became_active":"2020-03-19T21:18:10.183810+0000","last_became_peered":"2020-03-19T21:18:10.183810+0000","last_unstale":"2020-03-19T21:18:29.245592+0000","last_undegraded":"2020-03-19T21:18:29.245592+0000","last_fullsized":"2020-03-19T21:18:29.245592+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:09.169195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:09.169195+0000","last_clean_scrub_stamp":"2020-03-19T21:18:09.169195+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"16'7","reported_seq":"25","reported_epoch":"23","state":"active+clean","last_fresh":"2020-03-19T21:18:29.245983+0000","last_change":"2020-03-19T21:18:13.196212+0000","last_active":"2020-03-19T21:18:29.245983+0000","last_peered":"2020-03-19T21:18:29.245983+0000","last_clean":"2020-03-19T21:18:29.245983+0000","last_became_active":"2020-03-19T21:18:13.195896+0000","last_became_peered":"2020-03-19T21:18:13.195896+0000","last_unstale":"2020-03-19T21:18:29.245983+0000","last_undegraded":"2020-03-19T21:18:29.245983+0000","last_fullsized":"2020-03-19T21:18:29.245983+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:12.182524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:12.182524+0000","last_clean_scrub_stamp":"2020-03-19T21:18:12.182524+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":566,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":"15","reported_epoch":"22","state":"active+clean","last_fresh":"2020-03-19T21:18:29.006837+0000","last_change":"2020-03-19T21:18:13.524166+0000","last_active":"2020-03-19T21:18:29.006837+0000","last_peered":"2020-03-19T21:18:29.006837+0000","last_clean":"2020-03-19T21:18:29.006837+0000","last_became_active":"2020-03-19T21:18:13.523806+0000","last_became_peered":"2020-03-19T21:18:13.523806+0000","last_unstale":"2020-03-19T21:18:29.006837+0000","last_undegraded":"2020-03-19T21:18:29.006837+0000","last_fullsized":"2020-03-19T21:18:29.006837+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:13.185694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:13.185694+0000","last_clean_scrub_stamp":"2020-03-19T21:18:13.185694+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":"20","reported_epoch":"23","state":"active+clean","last_fresh":"2020-03-19T21:18:29.245663+0000","last_change":"2020-03-19T21:18:10.184210+0000","last_active":"2020-03-19T21:18:29.245663+0000","last_peered":"2020-03-19T21:18:29.245663+0000","last_clean":"2020-03-19T21:18:29.245663+0000","last_became_active":"2020-03-19T21:18:10.183788+0000","last_became_peered":"2020-03-19T21:18:10.183788+0000","last_unstale":"2020-03-19T21:18:29.245663+0000","last_undegraded":"2020-03-19T21:18:29.245663+0000","last_fullsized":"2020-03-19T21:18:29.245663+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:09.169195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:09.169195+0000","last_clean_scrub_stamp":"2020-03-19T21:18:09.169195+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"16'8","reported_seq":"25","reported_epoch":"23","state":"active+clean","last_fresh":"2020-03-19T21:18:30.007670+0000","last_change":"2020-03-19T21:18:13.197291+0000","last_active":"2020-03-19T21:18:30.007670+0000","last_peered":"2020-03-19T21:18:30.007670+0000","last_clean":"2020-03-19T21:18:30.007670+0000","last_became_active":"2020-03-19T21:18:13.196823+0000","last_became_peered":"2020-03-19T21:18:13.196823+0000","last_unstale":"2020-03-19T21:18:30.007670+0000","last_undegraded":"2020-03-19T21:18:30.007670+0000","last_fullsized":"2020-03-19T21:18:30.007670+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:12.182524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:12.182524+0000","last_clean_scrub_stamp":"2020-03-19T21:18:12.182524+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":"17","reported_epoch":"23","state":"active+clean","last_fresh":"2020-03-19T21:18:29.245702+0000","last_change":"2020-03-19T21:18:13.528239+0000","last_active":"2020-03-19T21:18:29.245702+0000","last_peered":"2020-03-19T21:18:29.245702+0000","last_clean":"2020-03-19T21:18:29.245702+0000","last_became_active":"2020-03-19T21:18:13.527916+0000","last_became_peered":"2020-03-19T21:18:13.527916+0000","last_unstale":"2020-03-19T21:18:29.245702+0000","last_undegraded":"2020-03-19T21:18:29.245702+0000","last_fullsized":"2020-03-19T21:18:29.245702+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:13.185694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:13.185694+0000","last_clean_scrub_stamp":"2020-03-19T21:18:13.185694+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":"18","reported_epoch":"22","state":"active+clean","last_fresh":"2020-03-19T21:18:29.006780+0000","last_change":"2020-03-19T21:18:10.187875+0000","last_active":"2020-03-19T21:18:29.006780+0000","last_peered":"2020-03-19T21:18:29.006780+0000","last_clean":"2020-03-19T21:18:29.006780+0000","last_became_active":"2020-03-19T21:18:10.187570+0000","last_became_peered":"2020-03-19T21:18:10.187570+0000","last_unstale":"2020-03-19T21:18:29.006780+0000","last_undegraded":"2020-03-19T21:18:29.006780+0000","last_fullsized":"2020-03-19T21:18:29.006780+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:09.169195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:09.169195+0000","last_clean_scrub_stamp":"2020-03-19T21:18:09.169195+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"17","reported_epoch":"23","state":"active+clean","last_fresh":"2020-03-19T21:18:30.008133+0000","last_change":"2020-03-19T21:18:13.197535+0000","last_active":"2020-03-19T21:18:30.008133+0000","last_peered":"2020-03-19T21:18:30.008133+0000","last_clean":"2020-03-19T21:18:30.008133+0000","last_became_active":"2020-03-19T21:18:13.197224+0000","last_became_peered":"2020-03-19T21:18:13.197224+0000","last_unstale":"2020-03-19T21:18:30.008133+0000","last_undegraded":"2020-03-19T21:18:30.008133+0000","last_fullsized":"2020-03-19T21:18:30.008133+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:12.182524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:12.182524+0000","last_clean_scrub_stamp":"2020-03-19T21:18:12.182524+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":"17","reported_epoch":"23","state":"active+clean","last_fresh":"2020-03-19T21:18:29.245873+0000","last_change":"2020-03-19T21:18:13.524218+0000","last_active":"2020-03-19T21:18:29.245873+0000","last_peered":"2020-03-19T21:18:29.245873+0000","last_clean":"2020-03-19T21:18:29.245873+0000","last_became_active":"2020-03-19T21:18:13.523902+0000","last_became_peered":"2020-03-19T21:18:13.523902+0000","last_unstale":"2020-03-19T21:18:29.245873+0000","last_undegraded":"2020-03-19T21:18:29.245873+0000","last_fullsized":"2020-03-19T21:18:29.245873+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:13.185694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:13.185694+0000","last_clean_scrub_stamp":"2020-03-19T21:18:13.185694+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":"20","reported_epoch":"23","state":"active+clean","last_fresh":"2020-03-19T21:18:29.245602+0000","last_change":"2020-03-19T21:18:10.184801+0000","last_active":"2020-03-19T21:18:29.245602+0000","last_peered":"2020-03-19T21:18:29.245602+0000","last_clean":"2020-03-19T21:18:29.245602+0000","last_became_active":"2020-03-19T21:18:10.184433+0000","last_became_peered":"2020-03-19T21:18:10.184433+0000","last_unstale":"2020-03-19T21:18:29.245602+0000","last_undegraded":"2020-03-19T21:18:29.245602+0000","last_fullsized":"2020-03-19T21:18:29.245602+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:09.169195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:09.169195+0000","last_clean_scrub_stamp":"2020-03-19T21:18:09.169195+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"16'5","reported_seq":"23","reported_epoch":"23","state":"active+clean","last_fresh":"2020-03-19T21:18:29.245548+0000","last_change":"2020-03-19T21:18:13.195202+0000","last_active":"2020-03-19T21:18:29.245548+0000","last_peered":"2020-03-19T21:18:29.245548+0000","last_clean":"2020-03-19T21:18:29.245548+0000","last_became_active":"2020-03-19T21:18:13.194849+0000","last_became_peered":"2020-03-19T21:18:13.194849+0000","last_unstale":"2020-03-19T21:18:29.245548+0000","last_undegraded":"2020-03-19T21:18:29.245548+0000","last_fullsized":"2020-03-19T21:18:29.245548+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:12.182524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:12.182524+0000","last_clean_scrub_stamp":"2020-03-19T21:18:12.182524+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":"15","reported_epoch":"22","state":"active+clean","last_fresh":"2020-03-19T21:18:29.006919+0000","last_change":"2020-03-19T21:18:13.523284+0000","last_active":"2020-03-19T21:18:29.006919+0000","last_peered":"2020-03-19T21:18:29.006919+0000","last_clean":"2020-03-19T21:18:29.006919+0000","last_became_active":"2020-03-19T21:18:13.522937+0000","last_became_peered":"2020-03-19T21:18:13.522937+0000","last_unstale":"2020-03-19T21:18:29.006919+0000","last_undegraded":"2020-03-19T21:18:29.006919+0000","last_fullsized":"2020-03-19T21:18:29.006919+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:13.185694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:13.185694+0000","last_clean_scrub_stamp":"2020-03-19T21:18:13.185694+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":"20","reported_epoch":"23","state":"active+clean","last_fresh":"2020-03-19T21:18:30.008123+0000","last_change":"2020-03-19T21:18:10.183717+0000","last_active":"2020-03-19T21:18:30.008123+0000","last_peered":"2020-03-19T21:18:30.008123+0000","last_clean":"2020-03-19T21:18:30.008123+0000","last_became_active":"2020-03-19T21:18:10.183404+0000","last_became_peered":"2020-03-19T21:18:10.183404+0000","last_unstale":"2020-03-19T21:18:30.008123+0000","last_undegraded":"2020-03-19T21:18:30.008123+0000","last_fullsized":"2020-03-19T21:18:30.008123+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:09.169195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:09.169195+0000","last_clean_scrub_stamp":"2020-03-19T21:18:09.169195+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"16'1","reported_seq":"17","reported_epoch":"22","state":"active+clean","last_fresh":"2020-03-19T21:18:29.006883+0000","last_change":"2020-03-19T21:18:13.198955+0000","last_active":"2020-03-19T21:18:29.006883+0000","last_peered":"2020-03-19T21:18:29.006883+0000","last_clean":"2020-03-19T21:18:29.006883+0000","last_became_active":"2020-03-19T21:18:13.198498+0000","last_became_peered":"2020-03-19T21:18:13.198498+0000","last_unstale":"2020-03-19T21:18:29.006883+0000","last_undegraded":"2020-03-19T21:18:29.006883+0000","last_fullsized":"2020-03-19T21:18:29.006883+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:12.182524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:12.182524+0000","last_clean_scrub_stamp":"2020-03-19T21:18:12.182524+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":"12","reported_epoch":"22","state":"active+clean","last_fresh":"2020-03-19T21:18:29.006783+0000","last_change":"2020-03-19T21:18:24.224701+0000","last_active":"2020-03-19T21:18:29.006783+0000","last_peered":"2020-03-19T21:18:29.006783+0000","last_clean":"2020-03-19T21:18:29.006783+0000","last_became_active":"2020-03-19T21:18:24.224328+0000","last_became_peered":"2020-03-19T21:18:24.224328+0000","last_unstale":"2020-03-19T21:18:29.006783+0000","last_undegraded":"2020-03-19T21:18:29.006783+0000","last_fullsized":"2020-03-19T21:18:29.006783+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:23.204329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:23.204329+0000","last_clean_scrub_stamp":"2020-03-19T21:18:23.204329+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":"17","reported_epoch":"23","state":"active+clean","last_fresh":"2020-03-19T21:18:29.245491+0000","last_change":"2020-03-19T21:18:13.524754+0000","last_active":"2020-03-19T21:18:29.245491+0000","last_peered":"2020-03-19T21:18:29.245491+0000","last_clean":"2020-03-19T21:18:29.245491+0000","last_became_active":"2020-03-19T21:18:13.524437+0000","last_became_peered":"2020-03-19T21:18:13.524437+0000","last_unstale":"2020-03-19T21:18:29.245491+0000","last_undegraded":"2020-03-19T21:18:29.245491+0000","last_fullsized":"2020-03-19T21:18:29.245491+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:13.185694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:13.185694+0000","last_clean_scrub_stamp":"2020-03-19T21:18:13.185694+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":"20","reported_epoch":"23","state":"active+clean","last_fresh":"2020-03-19T21:18:29.245786+0000","last_change":"2020-03-19T21:18:10.184911+0000","last_active":"2020-03-19T21:18:29.245786+0000","last_peered":"2020-03-19T21:18:29.245786+0000","last_clean":"2020-03-19T21:18:29.245786+0000","last_became_active":"2020-03-19T21:18:10.184549+0000","last_became_peered":"2020-03-19T21:18:10.184549+0000","last_unstale":"2020-03-19T21:18:29.245786+0000","last_undegraded":"2020-03-19T21:18:29.245786+0000","last_fullsized":"2020-03-19T21:18:29.245786+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:09.169195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:09.169195+0000","last_clean_scrub_stamp":"2020-03-19T21:18:09.169195+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"16'6","reported_seq":"24","reported_epoch":"23","state":"active+clean","last_fresh":"2020-03-19T21:18:29.245726+0000","last_change":"2020-03-19T21:18:13.196469+0000","last_active":"2020-03-19T21:18:29.245726+0000","last_peered":"2020-03-19T21:18:29.245726+0000","last_clean":"2020-03-19T21:18:29.245726+0000","last_became_active":"2020-03-19T21:18:13.195947+0000","last_became_peered":"2020-03-19T21:18:13.195947+0000","last_unstale":"2020-03-19T21:18:29.245726+0000","last_undegraded":"2020-03-19T21:18:29.245726+0000","last_fullsized":"2020-03-19T21:18:29.245726+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:12.182524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:12.182524+0000","last_clean_scrub_stamp":"2020-03-19T21:18:12.182524+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":160,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":"14","reported_epoch":"23","state":"active+clean","last_fresh":"2020-03-19T21:18:29.245638+0000","last_change":"2020-03-19T21:18:24.223995+0000","last_active":"2020-03-19T21:18:29.245638+0000","last_peered":"2020-03-19T21:18:29.245638+0000","last_clean":"2020-03-19T21:18:29.245638+0000","last_became_active":"2020-03-19T21:18:24.223594+0000","last_became_peered":"2020-03-19T21:18:24.223594+0000","last_unstale":"2020-03-19T21:18:29.245638+0000","last_undegraded":"2020-03-19T21:18:29.245638+0000","last_fullsized":"2020-03-19T21:18:29.245638+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:23.204329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:23.204329+0000","last_clean_scrub_stamp":"2020-03-19T21:18:23.204329+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":"16","reported_epoch":"23","state":"active+clean","last_fresh":"2020-03-19T21:18:30.007327+0000","last_change":"2020-03-19T21:18:13.525236+0000","last_active":"2020-03-19T21:18:30.007327+0000","last_peered":"2020-03-19T21:18:30.007327+0000","last_clean":"2020-03-19T21:18:30.007327+0000","last_became_active":"2020-03-19T21:18:13.524938+0000","last_became_peered":"2020-03-19T21:18:13.524938+0000","last_unstale":"2020-03-19T21:18:30.007327+0000","last_undegraded":"2020-03-19T21:18:30.007327+0000","last_fullsized":"2020-03-19T21:18:30.007327+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:13.185694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:13.185694+0000","last_clean_scrub_stamp":"2020-03-19T21:18:13.185694+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":"20","reported_epoch":"23","state":"active+clean","last_fresh":"2020-03-19T21:18:29.245896+0000","last_change":"2020-03-19T21:18:10.182075+0000","last_active":"2020-03-19T21:18:29.245896+0000","last_peered":"2020-03-19T21:18:29.245896+0000","last_clean":"2020-03-19T21:18:29.245896+0000","last_became_active":"2020-03-19T21:18:10.181738+0000","last_became_peered":"2020-03-19T21:18:10.181738+0000","last_unstale":"2020-03-19T21:18:29.245896+0000","last_undegraded":"2020-03-19T21:18:29.245896+0000","last_fullsized":"2020-03-19T21:18:29.245896+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:09.169195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:09.169195+0000","last_clean_scrub_stamp":"2020-03-19T21:18:09.169195+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"16'8","reported_seq":"26","reported_epoch":"23","state":"active+clean","last_fresh":"2020-03-19T21:18:29.245728+0000","last_change":"2020-03-19T21:18:13.197376+0000","last_active":"2020-03-19T21:18:29.245728+0000","last_peered":"2020-03-19T21:18:29.245728+0000","last_clean":"2020-03-19T21:18:29.245728+0000","last_became_active":"2020-03-19T21:18:13.196089+0000","last_became_peered":"2020-03-19T21:18:13.196089+0000","last_unstale":"2020-03-19T21:18:29.245728+0000","last_undegraded":"2020-03-19T21:18:29.245728+0000","last_fullsized":"2020-03-19T21:18:29.245728+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:12.182524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:12.182524+0000","last_clean_scrub_stamp":"2020-03-19T21:18:12.182524+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1526,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":4,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":8,"acting":8,"num_store_stats":3},{"poolid":3,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":2286,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738377,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":1048780,"kb_used_data":196,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":103808820,"statfs":{"total":107374182400,"available":106300231680,"internally_reserved":1073741824,"allocated":200704,"data_stored":73454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57999999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68300000000000005}]}]},{"osd":0,"up_from":8,"seq":34359738376,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":1048796,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":103808804,"statfs":{"total":107374182400,"available":106300215296,"internally_reserved":1073741824,"allocated":217088,"data_stored":71341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60699999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54900000000000004}]}]},{"osd":1,"up_from":8,"seq":34359738376,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":1048808,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":103808792,"statfs":{"total":107374182400,"available":106300203008,"internally_reserved":1073741824,"allocated":229376,"data_stored":75706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48499999999999999}]}]}]}} 2020-03-19T21:18:34.329 INFO:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-03-19T21:18:34.435 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.432+0000 7f516f655700 1 Processor -- start 2020-03-19T21:18:34.436 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.432+0000 7f516f655700 1 -- start start 2020-03-19T21:18:34.436 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.433+0000 7f516f655700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f5168066760 0x7f5168066b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T21:18:34.436 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.433+0000 7f516f655700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51680912a0 con 0x7f5168090890 2020-03-19T21:18:34.436 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.433+0000 7f516f655700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f516808ddb0 con 0x7f5168066760 2020-03-19T21:18:34.436 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.433+0000 7f516d3f1700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f5168066760 0x7f5168066b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:34.436 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.433+0000 7f516cbf0700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f5168090890 0x7f516808bef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:54064/0 (socket says 172.21.15.170:54064) 2020-03-19T21:18:34.436 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.433+0000 7f516cbf0700 1 -- 172.21.15.170:0/3771551462 learned_addr learned my addr 172.21.15.170:0/3771551462 (peer_addr_for_me v1:172.21.15.170:0/0) 2020-03-19T21:18:34.436 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.433+0000 7f516d3f1700 1 -- 172.21.15.170:0/3771551462 >> v1:172.21.15.170:6789/0 conn(0x7f5168090890 legacy=0x7f516808bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:34.437 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.433+0000 7f516d3f1700 1 -- 172.21.15.170:0/3771551462 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f516808d1b0 con 0x7f5168066760 2020-03-19T21:18:34.437 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.433+0000 7f516d3f1700 1 --2- 172.21.15.170:0/3771551462 >> v2:172.21.15.170:3300/0 conn(0x7f5168066760 0x7f5168066b40 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f51640010d0 tx=0x7f5158007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:34.437 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.433+0000 7f516d3f1700 1 --2- 172.21.15.170:0/3771551462 >> v2:172.21.15.170:3300/0 conn(0x7f5168066760 0x7f5168066b40 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f51640010d0 tx=0x7f5158007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:34.437 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.433+0000 7f516d3f1700 1 --2- 172.21.15.170:0/3771551462 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5168066760 0x7f5168066b40 secure :-1 s=READY pgs=267 cs=0 l=1 rx=0x7f51640010d0 tx=0x7f5158007a40).ready entity=mon.0 client_cookie=95e63438a1466722 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:34.437 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.433+0000 7f516d3f1700 1 --2- 172.21.15.170:0/3771551462 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5168066760 0x7f5168066b40 secure :-1 s=THROTTLE_DONE pgs=267 cs=0 l=1 rx=0x7f51640010d0 tx=0x7f5158007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:34.437 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.433+0000 7f515ffff700 1 -- 172.21.15.170:0/3771551462 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f5158008bb0 con 0x7f5168066760 2020-03-19T21:18:34.437 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.433+0000 7f516d3f1700 1 --2- 172.21.15.170:0/3771551462 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5168066760 0x7f5168066b40 secure :-1 s=THROTTLE_DONE pgs=267 cs=0 l=1 rx=0x7f51640010d0 tx=0x7f5158007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:34.438 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.433+0000 7f515ffff700 1 -- 172.21.15.170:0/3771551462 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5158008d10 con 0x7f5168066760 2020-03-19T21:18:34.438 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.433+0000 7f516d3f1700 1 --2- 172.21.15.170:0/3771551462 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5168066760 0x7f5168066b40 secure :-1 s=THROTTLE_DONE pgs=267 cs=0 l=1 rx=0x7f51640010d0 tx=0x7f5158007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:34.438 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.433+0000 7f515ffff700 1 -- 172.21.15.170:0/3771551462 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f5158011890 con 0x7f5168066760 2020-03-19T21:18:34.438 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.433+0000 7f516f655700 1 -- 172.21.15.170:0/3771551462 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5168066760 msgr2=0x7f5168066b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:34.438 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.433+0000 7f516f655700 1 --2- 172.21.15.170:0/3771551462 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5168066760 0x7f5168066b40 secure :-1 s=READY pgs=267 cs=0 l=1 rx=0x7f51640010d0 tx=0x7f5158007a40).stop 2020-03-19T21:18:34.438 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.434+0000 7f516f655700 1 -- 172.21.15.170:0/3771551462 shutdown_connections 2020-03-19T21:18:34.438 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.434+0000 7f516f655700 1 --2- 172.21.15.170:0/3771551462 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5168066760 0x7f5168066b40 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:34.438 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.434+0000 7f516f655700 1 -- 172.21.15.170:0/3771551462 >> 172.21.15.170:0/3771551462 conn(0x7f5168091540 msgr2=0x7f5168091950 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:34.439 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.434+0000 7f516f655700 1 -- 172.21.15.170:0/3771551462 shutdown_connections 2020-03-19T21:18:34.439 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.434+0000 7f516f655700 1 -- 172.21.15.170:0/3771551462 wait complete. 2020-03-19T21:18:34.441 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.435+0000 7f516f655700 1 Processor -- start 2020-03-19T21:18:34.442 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.435+0000 7f516f655700 1 -- start start 2020-03-19T21:18:34.442 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.435+0000 7f516f655700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f5168090890 0x7f5168061550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:34.442 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.435+0000 7f516f655700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5168122050 con 0x7f5168066760 2020-03-19T21:18:34.442 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.435+0000 7f516f655700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51680607f0 con 0x7f5168090890 2020-03-19T21:18:34.442 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.435+0000 7f516d3f1700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f5168066760 0x7f5168060e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:54066/0 (socket says 172.21.15.170:54066) 2020-03-19T21:18:34.442 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.435+0000 7f516d3f1700 1 -- 172.21.15.170:0/2760759431 learned_addr learned my addr 172.21.15.170:0/2760759431 (peer_addr_for_me v1:172.21.15.170:0/0) 2020-03-19T21:18:34.442 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.435+0000 7f516cbf0700 1 --2- 172.21.15.170:0/2760759431 >> v2:172.21.15.170:3300/0 conn(0x7f5168090890 0x7f5168061550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:34.443 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.435+0000 7f515e7fc700 1 -- 172.21.15.170:0/2760759431 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2252475480 0 0) 0x7f5168122050 con 0x7f5168066760 2020-03-19T21:18:34.443 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.435+0000 7f515e7fc700 1 -- 172.21.15.170:0/2760759431 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5150002fe0 con 0x7f5168066760 2020-03-19T21:18:34.443 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.436+0000 7f516cbf0700 1 -- 172.21.15.170:0/2760759431 >> v1:172.21.15.170:6789/0 conn(0x7f5168066760 legacy=0x7f5168060e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:34.443 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.436+0000 7f516cbf0700 1 -- 172.21.15.170:0/2760759431 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5168121680 con 0x7f5168090890 2020-03-19T21:18:34.443 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.436+0000 7f516cbf0700 1 --2- 172.21.15.170:0/2760759431 >> v2:172.21.15.170:3300/0 conn(0x7f5168090890 0x7f5168061550 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f5164001190 tx=0x7f51640109f0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:34.443 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.436+0000 7f516cbf0700 1 --2- 172.21.15.170:0/2760759431 >> v2:172.21.15.170:3300/0 conn(0x7f5168090890 0x7f5168061550 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f5164001190 tx=0x7f51640109f0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:34.443 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.436+0000 7f516cbf0700 1 --2- 172.21.15.170:0/2760759431 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5168090890 0x7f5168061550 secure :-1 s=READY pgs=269 cs=0 l=1 rx=0x7f5164001190 tx=0x7f51640109f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:34.444 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.436+0000 7f516cbf0700 1 --2- 172.21.15.170:0/2760759431 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5168090890 0x7f5168061550 secure :-1 s=THROTTLE_DONE pgs=269 cs=0 l=1 rx=0x7f5164001190 tx=0x7f51640109f0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:34.444 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.436+0000 7f515e7fc700 1 -- 172.21.15.170:0/2760759431 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f51640104b0 con 0x7f5168090890 2020-03-19T21:18:34.444 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.436+0000 7f516cbf0700 1 --2- 172.21.15.170:0/2760759431 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5168090890 0x7f5168061550 secure :-1 s=THROTTLE_DONE pgs=269 cs=0 l=1 rx=0x7f5164001190 tx=0x7f51640109f0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:34.444 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.436+0000 7f515e7fc700 1 -- 172.21.15.170:0/2760759431 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f516400c950 con 0x7f5168090890 2020-03-19T21:18:34.444 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.436+0000 7f516cbf0700 1 --2- 172.21.15.170:0/2760759431 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5168090890 0x7f5168061550 secure :-1 s=THROTTLE_DONE pgs=269 cs=0 l=1 rx=0x7f5164001190 tx=0x7f51640109f0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:34.444 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.436+0000 7f516f655700 1 -- 172.21.15.170:0/2760759431 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5168062060 con 0x7f5168090890 2020-03-19T21:18:34.444 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.436+0000 7f515e7fc700 1 -- 172.21.15.170:0/2760759431 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f516400cc70 con 0x7f5168090890 2020-03-19T21:18:34.444 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.436+0000 7f516f655700 1 -- 172.21.15.170:0/2760759431 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5168124ab0 con 0x7f5168090890 2020-03-19T21:18:34.445 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.436+0000 7f516cbf0700 1 --2- 172.21.15.170:0/2760759431 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5168090890 0x7f5168061550 secure :-1 s=THROTTLE_DONE pgs=269 cs=0 l=1 rx=0x7f5164001190 tx=0x7f51640109f0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:34.445 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.436+0000 7f515e7fc700 1 -- 172.21.15.170:0/2760759431 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 4) v1 ==== 29603+0+0 (secure 0 0 0) 0x7f51640055c0 con 0x7f5168090890 2020-03-19T21:18:34.445 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.437+0000 7f515e7fc700 1 --2- 172.21.15.170:0/2760759431 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f5150028770 0x7f515002ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:34.445 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.437+0000 7f516d3f1700 1 --2- 172.21.15.170:0/2760759431 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f5150028770 0x7f515002ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:34.445 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.437+0000 7f516d3f1700 1 --2- 172.21.15.170:0/2760759431 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f5150028770 0x7f515002ac30 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f5158000b80 tx=0x7f5158007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:34.445 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.437+0000 7f516d3f1700 1 --2- 172.21.15.170:0/2760759431 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f5150028770 0x7f515002ac30 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f5158000b80 tx=0x7f5158007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:34.445 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.437+0000 7f516d3f1700 1 --2- 172.21.15.170:0/2760759431 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f5150028770 0x7f515002ac30 secure :-1 s=READY pgs=66 cs=0 l=1 rx=0x7f5158000b80 tx=0x7f5158007a40).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:34.446 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.437+0000 7f516f655700 1 -- 172.21.15.170:0/2760759431 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5168120fb0 con 0x7f5168090890 2020-03-19T21:18:34.446 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.437+0000 7f516cbf0700 1 --2- 172.21.15.170:0/2760759431 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5168090890 0x7f5168061550 secure :-1 s=THROTTLE_DONE pgs=269 cs=0 l=1 rx=0x7f5164001190 tx=0x7f51640109f0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:34.446 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.437+0000 7f515e7fc700 1 -- 172.21.15.170:0/2760759431 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 3786+0+0 (secure 0 0 0) 0x7f51640113e0 con 0x7f5168090890 2020-03-19T21:18:34.446 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.441+0000 7f516cbf0700 1 --2- 172.21.15.170:0/2760759431 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5168090890 0x7f5168061550 secure :-1 s=THROTTLE_DONE pgs=269 cs=0 l=1 rx=0x7f5164001190 tx=0x7f51640109f0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:34.446 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.441+0000 7f515e7fc700 1 -- 172.21.15.170:0/2760759431 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+126772 (secure 0 0 0) 0x7f51640080c0 con 0x7f5168090890 2020-03-19T21:18:34.579 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.578+0000 7f516f655700 1 -- 172.21.15.170:0/2760759431 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5168056db0 con 0x7f5168090890 2020-03-19T21:18:34.580 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.579+0000 7f516cbf0700 1 --2- 172.21.15.170:0/2760759431 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5168090890 0x7f5168061550 secure :-1 s=THROTTLE_DONE pgs=269 cs=0 l=1 rx=0x7f5164001190 tx=0x7f51640109f0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:34.580 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.579+0000 7f515e7fc700 1 -- 172.21.15.170:0/2760759431 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+10376 (secure 0 0 0) 0x7f51640100b0 con 0x7f5168090890 2020-03-19T21:18:34.580 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:18:34.580 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":24,"fsid":"12d0a33b-6b40-4398-90b4-6b5df1da388e","created":"2020-03-19T21:17:59.135747+0000","modified":"2020-03-19T21:18:30.247068+0000","last_up_change":"2020-03-19T21:18:05.158515+0000","last_in_change":"2020-03-19T21:18:05.158515+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":3,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2020-03-19T21:18:08.391604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2020-03-19T21:18:11.286536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2020-03-19T21:18:12.597110+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"20","last_force_op_resend_prenautilus":"20","last_force_op_resend_preluminous":"20","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":18,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[4],"tier_of":-1,"read_tier":4,"write_tier":4,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":4,"pool_name":"data_cache","create_time":"2020-03-19T21:18:22.941000+0000","flags":9,"flags_names":"hashpspool,incomplete_clones","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":4,"pg_placement_num":4,"pg_placement_num_target":4,"pg_num_target":4,"pg_num_pending":4,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"20","last_force_op_resend_prenautilus":"20","last_force_op_resend_preluminous":"20","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":18,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":3,"read_tier":-1,"write_tier":-1,"cache_mode":"writeback","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"bloom","false_positive_probability":0.050000000000000003,"target_size":0,"seed":0},"hit_set_period":3600,"hit_set_count":8,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[1000000,1000000,1000000,1000000,1000000,1000000,1000000,1000000],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{}}],"osds":[{"osd":0,"uuid":"32ca3517-1077-4102-9aa4-edfcebe8e3d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":20680},{"type":"v1","addr":"172.21.15.170:6801","nonce":20680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":20680},{"type":"v1","addr":"172.21.15.170:6803","nonce":20680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":20680},{"type":"v1","addr":"172.21.15.170:6807","nonce":20680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":20680},{"type":"v1","addr":"172.21.15.170:6805","nonce":20680}]},"public_addr":"172.21.15.170:6801/20680","cluster_addr":"172.21.15.170:6803/20680","heartbeat_back_addr":"172.21.15.170:6807/20680","heartbeat_front_addr":"172.21.15.170:6805/20680","state":["exists","up"]},{"osd":1,"uuid":"470df64b-60bc-4f5b-9b53-64495cd7c5e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":20681},{"type":"v1","addr":"172.21.15.170:6809","nonce":20681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":20681},{"type":"v1","addr":"172.21.15.170:6811","nonce":20681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":20681},{"type":"v1","addr":"172.21.15.170:6815","nonce":20681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":20681},{"type":"v1","addr":"172.21.15.170:6813","nonce":20681}]},"public_addr":"172.21.15.170:6809/20681","cluster_addr":"172.21.15.170:6811/20681","heartbeat_back_addr":"172.21.15.170:6815/20681","heartbeat_front_addr":"172.21.15.170:6813/20681","state":["exists","up"]},{"osd":2,"uuid":"ccfb9d82-b759-4ff4-8ddf-4f1d741bac5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":20682},{"type":"v1","addr":"172.21.15.170:6817","nonce":20682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":20682},{"type":"v1","addr":"172.21.15.170:6819","nonce":20682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":20682},{"type":"v1","addr":"172.21.15.170:6823","nonce":20682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":20682},{"type":"v1","addr":"172.21.15.170:6821","nonce":20682}]},"public_addr":"172.21.15.170:6817/20682","cluster_addr":"172.21.15.170:6819/20682","heartbeat_back_addr":"172.21.15.170:6823/20682","heartbeat_front_addr":"172.21.15.170:6821/20682","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2020-03-19T21:18:03.177293+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2020-03-19T21:18:03.263179+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2020-03-19T21:18:03.183223+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-03-19T21:18:34.582 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.580+0000 7f516f655700 1 -- 172.21.15.170:0/2760759431 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f5150028770 msgr2=0x7f515002ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:34.582 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.580+0000 7f516f655700 1 --2- 172.21.15.170:0/2760759431 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f5150028770 0x7f515002ac30 secure :-1 s=READY pgs=66 cs=0 l=1 rx=0x7f5158000b80 tx=0x7f5158007a40).stop 2020-03-19T21:18:34.582 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.580+0000 7f516f655700 1 -- 172.21.15.170:0/2760759431 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5168090890 msgr2=0x7f5168061550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:34.582 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.580+0000 7f516f655700 1 --2- 172.21.15.170:0/2760759431 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5168090890 0x7f5168061550 secure :-1 s=READY pgs=269 cs=0 l=1 rx=0x7f5164001190 tx=0x7f51640109f0).stop 2020-03-19T21:18:34.582 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.580+0000 7f516f655700 1 -- 172.21.15.170:0/2760759431 shutdown_connections 2020-03-19T21:18:34.582 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.580+0000 7f516f655700 1 --2- 172.21.15.170:0/2760759431 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f5150028770 0x7f515002ac30 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:34.583 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.580+0000 7f516f655700 1 --2- 172.21.15.170:0/2760759431 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5168090890 0x7f5168061550 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:34.583 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.580+0000 7f516f655700 1 -- 172.21.15.170:0/2760759431 >> 172.21.15.170:0/2760759431 conn(0x7f5168091540 msgr2=0x7f516808f440 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:34.583 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.580+0000 7f516f655700 1 -- 172.21.15.170:0/2760759431 shutdown_connections 2020-03-19T21:18:34.583 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:34.580+0000 7f516f655700 1 -- 172.21.15.170:0/2760759431 wait complete. 2020-03-19T21:18:35.593 INFO:tasks.ceph:Scrubbing osd.0 2020-03-19T21:18:35.593 INFO:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early tell osd.0 config set osd_debug_deep_scrub_sleep 0 2020-03-19T21:18:35.714 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.713+0000 7f577759e700 1 Processor -- start 2020-03-19T21:18:35.714 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.713+0000 7f577759e700 1 -- start start 2020-03-19T21:18:35.717 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.714+0000 7f577759e700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f577808e5c0 0x7f5778122960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T21:18:35.717 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.714+0000 7f577759e700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5778123270 con 0x7f5778122dc0 2020-03-19T21:18:35.717 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.714+0000 7f577759e700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57781223b0 con 0x7f577808e5c0 2020-03-19T21:18:35.717 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.714+0000 7f577659c700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f577808e5c0 0x7f5778122960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:35.718 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.714+0000 7f5775d9b700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f5778122dc0 0x7f5778060070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:54074/0 (socket says 172.21.15.170:54074) 2020-03-19T21:18:35.718 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.714+0000 7f5775d9b700 1 -- 172.21.15.170:0/971774857 learned_addr learned my addr 172.21.15.170:0/971774857 (peer_addr_for_me v1:172.21.15.170:0/0) 2020-03-19T21:18:35.718 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.714+0000 7f577659c700 1 -- 172.21.15.170:0/971774857 >> v1:172.21.15.170:6789/0 conn(0x7f5778122dc0 legacy=0x7f5778060070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:35.718 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.714+0000 7f577559a700 1 -- 172.21.15.170:0/971774857 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1794550358 0 0) 0x7f5778123270 con 0x7f5778122dc0 2020-03-19T21:18:35.718 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.714+0000 7f577659c700 1 -- 172.21.15.170:0/971774857 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5778091f00 con 0x7f577808e5c0 2020-03-19T21:18:35.718 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.714+0000 7f577659c700 1 --2- 172.21.15.170:0/971774857 >> v2:172.21.15.170:3300/0 conn(0x7f577808e5c0 0x7f5778122960 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f576c002f30 tx=0x7f576c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:35.718 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.714+0000 7f577659c700 1 --2- 172.21.15.170:0/971774857 >> v2:172.21.15.170:3300/0 conn(0x7f577808e5c0 0x7f5778122960 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f576c002f30 tx=0x7f576c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:35.719 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.714+0000 7f577659c700 1 --2- 172.21.15.170:0/971774857 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f577808e5c0 0x7f5778122960 secure :-1 s=READY pgs=271 cs=0 l=1 rx=0x7f576c002f30 tx=0x7f576c007a40).ready entity=mon.0 client_cookie=6e07eb045b8ef5e9 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:35.719 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.715+0000 7f577659c700 1 --2- 172.21.15.170:0/971774857 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f577808e5c0 0x7f5778122960 secure :-1 s=THROTTLE_DONE pgs=271 cs=0 l=1 rx=0x7f576c002f30 tx=0x7f576c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:35.719 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.715+0000 7f577559a700 1 -- 172.21.15.170:0/971774857 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f576c008e80 con 0x7f577808e5c0 2020-03-19T21:18:35.719 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.715+0000 7f577659c700 1 --2- 172.21.15.170:0/971774857 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f577808e5c0 0x7f5778122960 secure :-1 s=THROTTLE_DONE pgs=271 cs=0 l=1 rx=0x7f576c002f30 tx=0x7f576c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:35.719 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.715+0000 7f577559a700 1 -- 172.21.15.170:0/971774857 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f576c0113f0 con 0x7f577808e5c0 2020-03-19T21:18:35.719 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.715+0000 7f577659c700 1 --2- 172.21.15.170:0/971774857 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f577808e5c0 0x7f5778122960 secure :-1 s=THROTTLE_DONE pgs=271 cs=0 l=1 rx=0x7f576c002f30 tx=0x7f576c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:35.719 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.715+0000 7f577559a700 1 -- 172.21.15.170:0/971774857 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f576c0116d0 con 0x7f577808e5c0 2020-03-19T21:18:35.720 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.715+0000 7f577759e700 1 -- 172.21.15.170:0/971774857 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f577808e5c0 msgr2=0x7f5778122960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:35.720 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.715+0000 7f577759e700 1 --2- 172.21.15.170:0/971774857 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f577808e5c0 0x7f5778122960 secure :-1 s=READY pgs=271 cs=0 l=1 rx=0x7f576c002f30 tx=0x7f576c007a40).stop 2020-03-19T21:18:35.720 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.715+0000 7f577759e700 1 -- 172.21.15.170:0/971774857 shutdown_connections 2020-03-19T21:18:35.720 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.715+0000 7f577759e700 1 --2- 172.21.15.170:0/971774857 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f577808e5c0 0x7f5778122960 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:35.720 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.715+0000 7f577759e700 1 -- 172.21.15.170:0/971774857 >> 172.21.15.170:0/971774857 conn(0x7f57780950e0 msgr2=0x7f5778091180 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:35.720 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.715+0000 7f577759e700 1 -- 172.21.15.170:0/971774857 shutdown_connections 2020-03-19T21:18:35.721 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.715+0000 7f577759e700 1 -- 172.21.15.170:0/971774857 wait complete. 2020-03-19T21:18:35.721 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.716+0000 7f577759e700 1 Processor -- start 2020-03-19T21:18:35.721 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.716+0000 7f577759e700 1 -- start start 2020-03-19T21:18:35.724 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.716+0000 7f577759e700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f577808e5c0 0x7f5778132670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:35.724 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.716+0000 7f577759e700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5778062d70 con 0x7f5778122dc0 2020-03-19T21:18:35.724 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.716+0000 7f577759e700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5778062ad0 con 0x7f577808e5c0 2020-03-19T21:18:35.724 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.716+0000 7f577659c700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f577808e5c0 0x7f5778132670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:35.725 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.716+0000 7f5775d9b700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f5778122dc0 0x7f5778132ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:54078/0 (socket says 172.21.15.170:54078) 2020-03-19T21:18:35.725 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.716+0000 7f5775d9b700 1 -- 172.21.15.170:0/3035053033 learned_addr learned my addr 172.21.15.170:0/3035053033 (peer_addr_for_me v1:172.21.15.170:0/0) 2020-03-19T21:18:35.725 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.717+0000 7f577659c700 1 -- 172.21.15.170:0/3035053033 >> v1:172.21.15.170:6789/0 conn(0x7f5778122dc0 legacy=0x7f5778132ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:35.725 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.717+0000 7f57677fe700 1 -- 172.21.15.170:0/3035053033 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 849752058 0 0) 0x7f5778062d70 con 0x7f5778122dc0 2020-03-19T21:18:35.725 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.717+0000 7f577659c700 1 -- 172.21.15.170:0/3035053033 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5778111360 con 0x7f577808e5c0 2020-03-19T21:18:35.725 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.717+0000 7f577659c700 1 --2- 172.21.15.170:0/3035053033 >> v2:172.21.15.170:3300/0 conn(0x7f577808e5c0 0x7f5778132670 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f576c002f30 tx=0x7f576c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:35.725 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.717+0000 7f577659c700 1 --2- 172.21.15.170:0/3035053033 >> v2:172.21.15.170:3300/0 conn(0x7f577808e5c0 0x7f5778132670 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f576c002f30 tx=0x7f576c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:35.726 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.717+0000 7f577659c700 1 --2- 172.21.15.170:0/3035053033 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f577808e5c0 0x7f5778132670 secure :-1 s=READY pgs=273 cs=0 l=1 rx=0x7f576c002f30 tx=0x7f576c007a40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:35.726 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.717+0000 7f577659c700 1 --2- 172.21.15.170:0/3035053033 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f577808e5c0 0x7f5778132670 secure :-1 s=THROTTLE_DONE pgs=273 cs=0 l=1 rx=0x7f576c002f30 tx=0x7f576c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:35.726 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.717+0000 7f57677fe700 1 -- 172.21.15.170:0/3035053033 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f576c011dc0 con 0x7f577808e5c0 2020-03-19T21:18:35.726 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.717+0000 7f577659c700 1 --2- 172.21.15.170:0/3035053033 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f577808e5c0 0x7f5778132670 secure :-1 s=THROTTLE_DONE pgs=273 cs=0 l=1 rx=0x7f576c002f30 tx=0x7f576c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:35.726 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.717+0000 7f577659c700 1 --2- 172.21.15.170:0/3035053033 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f577808e5c0 0x7f5778132670 secure :-1 s=THROTTLE_DONE pgs=273 cs=0 l=1 rx=0x7f576c002f30 tx=0x7f576c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:35.726 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.717+0000 7f57677fe700 1 -- 172.21.15.170:0/3035053033 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f576c007150 con 0x7f577808e5c0 2020-03-19T21:18:35.726 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.717+0000 7f577759e700 1 -- 172.21.15.170:0/3035053033 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5778063520 con 0x7f577808e5c0 2020-03-19T21:18:35.727 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.717+0000 7f57677fe700 1 -- 172.21.15.170:0/3035053033 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f576c0073f0 con 0x7f577808e5c0 2020-03-19T21:18:35.727 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.717+0000 7f577759e700 1 -- 172.21.15.170:0/3035053033 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5778110770 con 0x7f577808e5c0 2020-03-19T21:18:35.727 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.717+0000 7f577659c700 1 --2- 172.21.15.170:0/3035053033 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f577808e5c0 0x7f5778132670 secure :-1 s=THROTTLE_DONE pgs=273 cs=0 l=1 rx=0x7f576c002f30 tx=0x7f576c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:35.727 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.718+0000 7f57677fe700 1 -- 172.21.15.170:0/3035053033 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 4) v1 ==== 29603+0+0 (secure 0 0 0) 0x7f576c0056c0 con 0x7f577808e5c0 2020-03-19T21:18:35.727 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.718+0000 7f577659c700 1 --2- 172.21.15.170:0/3035053033 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f577808e5c0 0x7f5778132670 secure :-1 s=THROTTLE_DONE pgs=273 cs=0 l=1 rx=0x7f576c002f30 tx=0x7f576c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:35.727 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.718+0000 7f57677fe700 1 --2- 172.21.15.170:0/3035053033 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f5754028230 0x7f575402a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:35.727 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.718+0000 7f57677fe700 1 -- 172.21.15.170:0/3035053033 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 3786+0+0 (secure 0 0 0) 0x7f576c034880 con 0x7f577808e5c0 2020-03-19T21:18:35.728 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.718+0000 7f5775d9b700 1 --2- 172.21.15.170:0/3035053033 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f5754028230 0x7f575402a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:35.728 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.718+0000 7f577759e700 1 --2- 172.21.15.170:0/3035053033 >> [v2:172.21.15.170:6800/20680,v1:172.21.15.170:6801/20680] conn(0x7f5778055b30 0x7f5778055f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:35.728 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.718+0000 7f577759e700 1 -- 172.21.15.170:0/3035053033 --> [v2:172.21.15.170:6800/20680,v1:172.21.15.170:6801/20680] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5778121e10 con 0x7f5778055b30 2020-03-19T21:18:35.728 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.718+0000 7f5775d9b700 1 --2- 172.21.15.170:0/3035053033 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f5754028230 0x7f575402a6f0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f5768005500 tx=0x7f5768001700).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:35.728 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.718+0000 7f5775d9b700 1 --2- 172.21.15.170:0/3035053033 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f5754028230 0x7f575402a6f0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f5768005500 tx=0x7f5768001700).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:35.728 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.718+0000 7f5775d9b700 1 --2- 172.21.15.170:0/3035053033 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f5754028230 0x7f575402a6f0 secure :-1 s=READY pgs=67 cs=0 l=1 rx=0x7f5768005500 tx=0x7f5768001700).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:35.728 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.718+0000 7f5776d9d700 1 --2- 172.21.15.170:0/3035053033 >> [v2:172.21.15.170:6800/20680,v1:172.21.15.170:6801/20680] conn(0x7f5778055b30 0x7f5778055f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:35.729 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.718+0000 7f5776d9d700 1 --2- 172.21.15.170:0/3035053033 >> [v2:172.21.15.170:6800/20680,v1:172.21.15.170:6801/20680] conn(0x7f5778055b30 0x7f5778055f10 crc :-1 s=READY pgs=7 cs=0 l=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:35.729 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.719+0000 7f57677fe700 1 -- 172.21.15.170:0/3035053033 <== osd.0 v2:172.21.15.170:6800/20680 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23288 (crc 0 0 0) 0x7f5778121e10 con 0x7f5778055b30 2020-03-19T21:18:35.736 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.735+0000 7f577759e700 1 -- 172.21.15.170:0/3035053033 --> [v2:172.21.15.170:6800/20680,v1:172.21.15.170:6801/20680] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f57780565b0 con 0x7f5778055b30 2020-03-19T21:18:35.738 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.736+0000 7f57677fe700 1 -- 172.21.15.170:0/3035053033 <== osd.0 v2:172.21.15.170:6800/20680 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f57780565b0 con 0x7f5778055b30 2020-03-19T21:18:35.738 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.736+0000 7f577759e700 1 -- 172.21.15.170:0/3035053033 >> [v2:172.21.15.170:6800/20680,v1:172.21.15.170:6801/20680] conn(0x7f5778055b30 msgr2=0x7f5778055f10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:35.738 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.736+0000 7f577759e700 1 --2- 172.21.15.170:0/3035053033 >> [v2:172.21.15.170:6800/20680,v1:172.21.15.170:6801/20680] conn(0x7f5778055b30 0x7f5778055f10 crc :-1 s=READY pgs=7 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:35.738 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.736+0000 7f577759e700 1 -- 172.21.15.170:0/3035053033 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f5754028230 msgr2=0x7f575402a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:35.738 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.736+0000 7f577759e700 1 --2- 172.21.15.170:0/3035053033 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f5754028230 0x7f575402a6f0 secure :-1 s=READY pgs=67 cs=0 l=1 rx=0x7f5768005500 tx=0x7f5768001700).stop 2020-03-19T21:18:35.738 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.736+0000 7f577759e700 1 -- 172.21.15.170:0/3035053033 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f577808e5c0 msgr2=0x7f5778132670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:35.738 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.736+0000 7f577759e700 1 --2- 172.21.15.170:0/3035053033 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f577808e5c0 0x7f5778132670 secure :-1 s=READY pgs=273 cs=0 l=1 rx=0x7f576c002f30 tx=0x7f576c007a40).stop 2020-03-19T21:18:35.739 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.736+0000 7f577759e700 1 -- 172.21.15.170:0/3035053033 shutdown_connections 2020-03-19T21:18:35.739 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.736+0000 7f577759e700 1 --2- 172.21.15.170:0/3035053033 >> [v2:172.21.15.170:6800/20680,v1:172.21.15.170:6801/20680] conn(0x7f5778055b30 0x7f5778055f10 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:35.739 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.736+0000 7f577759e700 1 --2- 172.21.15.170:0/3035053033 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f5754028230 0x7f575402a6f0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:35.739 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.736+0000 7f577759e700 1 --2- 172.21.15.170:0/3035053033 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f577808e5c0 0x7f5778132670 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:35.739 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.736+0000 7f577759e700 1 -- 172.21.15.170:0/3035053033 >> 172.21.15.170:0/3035053033 conn(0x7f57780950e0 msgr2=0x7f57781201e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:35.739 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.736+0000 7f577759e700 1 -- 172.21.15.170:0/3035053033 shutdown_connections 2020-03-19T21:18:35.739 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.736+0000 7f577759e700 1 -- 172.21.15.170:0/3035053033 wait complete. 2020-03-19T21:18:35.740 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2020-03-19T21:18:35.740 INFO:teuthology.orchestra.run.smithi170.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2020-03-19T21:18:35.740 INFO:teuthology.orchestra.run.smithi170.stdout:} 2020-03-19T21:18:35.747 INFO:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd deep-scrub 0 2020-03-19T21:18:35.852 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.851+0000 7f6099dfa700 1 Processor -- start 2020-03-19T21:18:35.853 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.851+0000 7f6099dfa700 1 -- start start 2020-03-19T21:18:35.858 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.852+0000 7f6099dfa700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f6094091eb0 0x7f609411f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T21:18:35.858 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.852+0000 7f6099dfa700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60940911c0 con 0x7f609411fab0 2020-03-19T21:18:35.858 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.852+0000 7f6099dfa700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6094067130 con 0x7f6094091eb0 2020-03-19T21:18:35.859 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.852+0000 7f60937fe700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f6094091eb0 0x7f609411f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:35.859 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.852+0000 7f6092ffd700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f609411fab0 0x7f6094060110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:54084/0 (socket says 172.21.15.170:54084) 2020-03-19T21:18:35.859 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.852+0000 7f6092ffd700 1 -- 172.21.15.170:0/2667119043 learned_addr learned my addr 172.21.15.170:0/2667119043 (peer_addr_for_me v1:172.21.15.170:0/0) 2020-03-19T21:18:35.859 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.852+0000 7f60937fe700 1 -- 172.21.15.170:0/2667119043 >> v1:172.21.15.170:6789/0 conn(0x7f609411fab0 legacy=0x7f6094060110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:35.859 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.852+0000 7f60937fe700 1 -- 172.21.15.170:0/2667119043 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f609411f340 con 0x7f6094091eb0 2020-03-19T21:18:35.859 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.852+0000 7f60927fc700 1 -- 172.21.15.170:0/2667119043 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2410421038 0 0) 0x7f60940911c0 con 0x7f609411fab0 2020-03-19T21:18:35.860 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.852+0000 7f60937fe700 1 --2- 172.21.15.170:0/2667119043 >> v2:172.21.15.170:3300/0 conn(0x7f6094091eb0 0x7f609411f650 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f6084002f30 tx=0x7f6084007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:35.860 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.852+0000 7f60937fe700 1 --2- 172.21.15.170:0/2667119043 >> v2:172.21.15.170:3300/0 conn(0x7f6094091eb0 0x7f609411f650 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f6084002f30 tx=0x7f6084007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:35.860 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.852+0000 7f60937fe700 1 --2- 172.21.15.170:0/2667119043 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6094091eb0 0x7f609411f650 secure :-1 s=READY pgs=275 cs=0 l=1 rx=0x7f6084002f30 tx=0x7f6084007a40).ready entity=mon.0 client_cookie=8071fd34f5875df8 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:35.860 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.852+0000 7f60937fe700 1 --2- 172.21.15.170:0/2667119043 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6094091eb0 0x7f609411f650 secure :-1 s=THROTTLE_DONE pgs=275 cs=0 l=1 rx=0x7f6084002f30 tx=0x7f6084007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:35.860 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.852+0000 7f60927fc700 1 -- 172.21.15.170:0/2667119043 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f6084008e80 con 0x7f6094091eb0 2020-03-19T21:18:35.860 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.852+0000 7f60937fe700 1 --2- 172.21.15.170:0/2667119043 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6094091eb0 0x7f609411f650 secure :-1 s=THROTTLE_DONE pgs=275 cs=0 l=1 rx=0x7f6084002f30 tx=0x7f6084007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:35.860 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.852+0000 7f60927fc700 1 -- 172.21.15.170:0/2667119043 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60840113f0 con 0x7f6094091eb0 2020-03-19T21:18:35.861 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.852+0000 7f60937fe700 1 --2- 172.21.15.170:0/2667119043 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6094091eb0 0x7f609411f650 secure :-1 s=THROTTLE_DONE pgs=275 cs=0 l=1 rx=0x7f6084002f30 tx=0x7f6084007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:35.861 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.852+0000 7f60927fc700 1 -- 172.21.15.170:0/2667119043 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f6084011690 con 0x7f6094091eb0 2020-03-19T21:18:35.861 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.852+0000 7f6099dfa700 1 -- 172.21.15.170:0/2667119043 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6094091eb0 msgr2=0x7f609411f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:35.861 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.852+0000 7f6099dfa700 1 --2- 172.21.15.170:0/2667119043 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6094091eb0 0x7f609411f650 secure :-1 s=READY pgs=275 cs=0 l=1 rx=0x7f6084002f30 tx=0x7f6084007a40).stop 2020-03-19T21:18:35.861 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.853+0000 7f6099dfa700 1 -- 172.21.15.170:0/2667119043 shutdown_connections 2020-03-19T21:18:35.861 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.853+0000 7f6099dfa700 1 --2- 172.21.15.170:0/2667119043 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6094091eb0 0x7f609411f650 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:35.862 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.853+0000 7f6099dfa700 1 -- 172.21.15.170:0/2667119043 >> 172.21.15.170:0/2667119043 conn(0x7f6094091390 msgr2=0x7f60940917a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:35.862 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.853+0000 7f6099dfa700 1 -- 172.21.15.170:0/2667119043 shutdown_connections 2020-03-19T21:18:35.862 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.853+0000 7f6099dfa700 1 -- 172.21.15.170:0/2667119043 wait complete. 2020-03-19T21:18:35.862 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.854+0000 7f6099dfa700 1 Processor -- start 2020-03-19T21:18:35.862 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.854+0000 7f6099dfa700 1 -- start start 2020-03-19T21:18:35.862 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.854+0000 7f6099dfa700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f609411fab0 0x7f609411bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:35.862 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.854+0000 7f6099dfa700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6094066a30 con 0x7f6094091eb0 2020-03-19T21:18:35.863 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.854+0000 7f6099dfa700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6094066c90 con 0x7f609411fab0 2020-03-19T21:18:35.863 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.854+0000 7f6092ffd700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f609411fab0 0x7f609411bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:35.863 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.854+0000 7f60937fe700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f6094091eb0 0x7f609411b820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:54088/0 (socket says 172.21.15.170:54088) 2020-03-19T21:18:35.863 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.854+0000 7f60937fe700 1 -- 172.21.15.170:0/3235968246 learned_addr learned my addr 172.21.15.170:0/3235968246 (peer_addr_for_me v1:172.21.15.170:0/0) 2020-03-19T21:18:35.863 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.855+0000 7f6092ffd700 1 -- 172.21.15.170:0/3235968246 >> v1:172.21.15.170:6789/0 conn(0x7f6094091eb0 legacy=0x7f609411b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:35.863 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.855+0000 7f6090ff9700 1 -- 172.21.15.170:0/3235968246 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1897624343 0 0) 0x7f6094066a30 con 0x7f6094091eb0 2020-03-19T21:18:35.863 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.855+0000 7f6092ffd700 1 -- 172.21.15.170:0/3235968246 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f609410b520 con 0x7f609411fab0 2020-03-19T21:18:35.864 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.855+0000 7f6092ffd700 1 --2- 172.21.15.170:0/3235968246 >> v2:172.21.15.170:3300/0 conn(0x7f609411fab0 0x7f609411bf30 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f6088000b80 tx=0x7f6088010910).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:35.864 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.855+0000 7f6092ffd700 1 --2- 172.21.15.170:0/3235968246 >> v2:172.21.15.170:3300/0 conn(0x7f609411fab0 0x7f609411bf30 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f6088000b80 tx=0x7f6088010910).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:35.864 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.855+0000 7f6092ffd700 1 --2- 172.21.15.170:0/3235968246 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f609411fab0 0x7f609411bf30 secure :-1 s=READY pgs=277 cs=0 l=1 rx=0x7f6088000b80 tx=0x7f6088010910).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:35.864 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.855+0000 7f6092ffd700 1 --2- 172.21.15.170:0/3235968246 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f609411fab0 0x7f609411bf30 secure :-1 s=THROTTLE_DONE pgs=277 cs=0 l=1 rx=0x7f6088000b80 tx=0x7f6088010910).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:35.864 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.855+0000 7f6090ff9700 1 -- 172.21.15.170:0/3235968246 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f6088005b20 con 0x7f609411fab0 2020-03-19T21:18:35.864 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.855+0000 7f6092ffd700 1 --2- 172.21.15.170:0/3235968246 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f609411fab0 0x7f609411bf30 secure :-1 s=THROTTLE_DONE pgs=277 cs=0 l=1 rx=0x7f6088000b80 tx=0x7f6088010910).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:35.865 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.855+0000 7f6092ffd700 1 --2- 172.21.15.170:0/3235968246 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f609411fab0 0x7f609411bf30 secure :-1 s=THROTTLE_DONE pgs=277 cs=0 l=1 rx=0x7f6088000b80 tx=0x7f6088010910).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:35.865 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.855+0000 7f6090ff9700 1 -- 172.21.15.170:0/3235968246 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6088005c80 con 0x7f609411fab0 2020-03-19T21:18:35.865 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.855+0000 7f6099dfa700 1 -- 172.21.15.170:0/3235968246 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6094064b40 con 0x7f609411fab0 2020-03-19T21:18:35.865 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.855+0000 7f6090ff9700 1 -- 172.21.15.170:0/3235968246 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f60880060c0 con 0x7f609411fab0 2020-03-19T21:18:35.866 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.855+0000 7f6099dfa700 1 -- 172.21.15.170:0/3235968246 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6094065460 con 0x7f609411fab0 2020-03-19T21:18:35.866 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.855+0000 7f6092ffd700 1 --2- 172.21.15.170:0/3235968246 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f609411fab0 0x7f609411bf30 secure :-1 s=THROTTLE_DONE pgs=277 cs=0 l=1 rx=0x7f6088000b80 tx=0x7f6088010910).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:35.866 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.856+0000 7f6090ff9700 1 -- 172.21.15.170:0/3235968246 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 4) v1 ==== 29603+0+0 (secure 0 0 0) 0x7f6088006330 con 0x7f609411fab0 2020-03-19T21:18:35.866 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.856+0000 7f6092ffd700 1 --2- 172.21.15.170:0/3235968246 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f609411fab0 0x7f609411bf30 secure :-1 s=THROTTLE_DONE pgs=277 cs=0 l=1 rx=0x7f6088000b80 tx=0x7f6088010910).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:35.866 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.856+0000 7f6090ff9700 1 --2- 172.21.15.170:0/3235968246 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f60740282b0 0x7f607402a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:35.866 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.856+0000 7f6090ff9700 1 -- 172.21.15.170:0/3235968246 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 3786+0+0 (secure 0 0 0) 0x7f6088033fb0 con 0x7f609411fab0 2020-03-19T21:18:35.866 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.856+0000 7f60937fe700 1 --2- 172.21.15.170:0/3235968246 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f60740282b0 0x7f607402a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:35.867 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.856+0000 7f6099dfa700 1 -- 172.21.15.170:0/3235968246 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f609410b260 con 0x7f609411fab0 2020-03-19T21:18:35.867 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.856+0000 7f60937fe700 1 --2- 172.21.15.170:0/3235968246 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f60740282b0 0x7f607402a770 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f60840101c0 tx=0x7f6084007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:35.867 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.856+0000 7f60937fe700 1 --2- 172.21.15.170:0/3235968246 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f60740282b0 0x7f607402a770 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f60840101c0 tx=0x7f6084007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:35.867 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.856+0000 7f60937fe700 1 --2- 172.21.15.170:0/3235968246 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f60740282b0 0x7f607402a770 secure :-1 s=READY pgs=68 cs=0 l=1 rx=0x7f60840101c0 tx=0x7f6084007a40).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:35.868 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.859+0000 7f6092ffd700 1 --2- 172.21.15.170:0/3235968246 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f609411fab0 0x7f609411bf30 secure :-1 s=THROTTLE_DONE pgs=277 cs=0 l=1 rx=0x7f6088000b80 tx=0x7f6088010910).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:35.868 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.859+0000 7f6090ff9700 1 -- 172.21.15.170:0/3235968246 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+126772 (secure 0 0 0) 0x7f6088006e60 con 0x7f609411fab0 2020-03-19T21:18:36.000 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.999+0000 7f6099dfa700 1 -- 172.21.15.170:0/3235968246 --> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f6094056db0 con 0x7f60740282b0 2020-03-19T21:18:36.000 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.999+0000 7f60937fe700 1 --2- 172.21.15.170:0/3235968246 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f60740282b0 0x7f607402a770 secure :-1 s=THROTTLE_DONE pgs=68 cs=0 l=1 rx=0x7f60840101c0 tx=0x7f6084007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:36.000 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:35.999+0000 7f6090ff9700 1 -- 172.21.15.170:0/3235968246 <== mgr.4102 v2:172.21.15.170:6824/20154 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f6094056db0 con 0x7f60740282b0 2020-03-19T21:18:36.002 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.000+0000 7f6099dfa700 1 -- 172.21.15.170:0/3235968246 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f60740282b0 msgr2=0x7f607402a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:36.002 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.000+0000 7f6099dfa700 1 --2- 172.21.15.170:0/3235968246 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f60740282b0 0x7f607402a770 secure :-1 s=READY pgs=68 cs=0 l=1 rx=0x7f60840101c0 tx=0x7f6084007a40).stop 2020-03-19T21:18:36.002 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.000+0000 7f6099dfa700 1 -- 172.21.15.170:0/3235968246 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f609411fab0 msgr2=0x7f609411bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:36.002 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.000+0000 7f6099dfa700 1 --2- 172.21.15.170:0/3235968246 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f609411fab0 0x7f609411bf30 secure :-1 s=READY pgs=277 cs=0 l=1 rx=0x7f6088000b80 tx=0x7f6088010910).stop 2020-03-19T21:18:36.002 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.000+0000 7f6099dfa700 1 -- 172.21.15.170:0/3235968246 shutdown_connections 2020-03-19T21:18:36.003 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.000+0000 7f6099dfa700 1 --2- 172.21.15.170:0/3235968246 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f60740282b0 0x7f607402a770 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:36.003 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.000+0000 7f6099dfa700 1 --2- 172.21.15.170:0/3235968246 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f609411fab0 0x7f609411bf30 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:36.003 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.000+0000 7f6099dfa700 1 -- 172.21.15.170:0/3235968246 >> 172.21.15.170:0/3235968246 conn(0x7f6094091390 msgr2=0x7f60940928f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:36.003 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.000+0000 7f6099dfa700 1 -- 172.21.15.170:0/3235968246 shutdown_connections 2020-03-19T21:18:36.003 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.000+0000 7f6099dfa700 1 -- 172.21.15.170:0/3235968246 wait complete. 2020-03-19T21:18:36.004 INFO:teuthology.orchestra.run.smithi170.stderr:instructed osd(s) 0 to deep-scrub 2020-03-19T21:18:36.011 INFO:tasks.ceph:Scrubbing osd.1 2020-03-19T21:18:36.012 INFO:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early tell osd.1 config set osd_debug_deep_scrub_sleep 0 2020-03-19T21:18:36.120 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.119+0000 7efe5043e700 1 Processor -- start 2020-03-19T21:18:36.121 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.119+0000 7efe5043e700 1 -- start start 2020-03-19T21:18:36.126 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.120+0000 7efe5043e700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7efe48125060 0x7efe48127450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T21:18:36.126 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.120+0000 7efe5043e700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe48060f10 con 0x7efe48065970 2020-03-19T21:18:36.126 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.120+0000 7efe5043e700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe48061490 con 0x7efe48125060 2020-03-19T21:18:36.126 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.120+0000 7efe4d9d9700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7efe48125060 0x7efe48127450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:36.127 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.120+0000 7efe4e1da700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7efe48065970 0x7efe48065d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:54094/0 (socket says 172.21.15.170:54094) 2020-03-19T21:18:36.127 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.120+0000 7efe4e1da700 1 -- 172.21.15.170:0/1560719772 learned_addr learned my addr 172.21.15.170:0/1560719772 (peer_addr_for_me v1:172.21.15.170:0/0) 2020-03-19T21:18:36.127 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.120+0000 7efe4d9d9700 1 -- 172.21.15.170:0/1560719772 >> v1:172.21.15.170:6789/0 conn(0x7efe48065970 legacy=0x7efe48065d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:36.127 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.120+0000 7efe4d9d9700 1 -- 172.21.15.170:0/1560719772 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe48061730 con 0x7efe48125060 2020-03-19T21:18:36.127 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.120+0000 7efe4d9d9700 1 --2- 172.21.15.170:0/1560719772 >> v2:172.21.15.170:3300/0 conn(0x7efe48125060 0x7efe48127450 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7efe44002f30 tx=0x7efe44007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:36.127 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.120+0000 7efe4d9d9700 1 --2- 172.21.15.170:0/1560719772 >> v2:172.21.15.170:3300/0 conn(0x7efe48125060 0x7efe48127450 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7efe44002f30 tx=0x7efe44007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:36.128 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.120+0000 7efe4d9d9700 1 --2- 172.21.15.170:0/1560719772 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe48125060 0x7efe48127450 secure :-1 s=READY pgs=279 cs=0 l=1 rx=0x7efe44002f30 tx=0x7efe44007a40).ready entity=mon.0 client_cookie=55af9e12a2a88756 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:36.128 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.121+0000 7efe4d9d9700 1 --2- 172.21.15.170:0/1560719772 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe48125060 0x7efe48127450 secure :-1 s=THROTTLE_DONE pgs=279 cs=0 l=1 rx=0x7efe44002f30 tx=0x7efe44007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:36.128 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.121+0000 7efe4d1d8700 1 -- 172.21.15.170:0/1560719772 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7efe440113f0 con 0x7efe48125060 2020-03-19T21:18:36.128 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.121+0000 7efe4d9d9700 1 --2- 172.21.15.170:0/1560719772 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe48125060 0x7efe48127450 secure :-1 s=THROTTLE_DONE pgs=279 cs=0 l=1 rx=0x7efe44002f30 tx=0x7efe44007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:36.128 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.121+0000 7efe4d1d8700 1 -- 172.21.15.170:0/1560719772 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe44011550 con 0x7efe48125060 2020-03-19T21:18:36.128 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.121+0000 7efe4d9d9700 1 --2- 172.21.15.170:0/1560719772 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe48125060 0x7efe48127450 secure :-1 s=THROTTLE_DONE pgs=279 cs=0 l=1 rx=0x7efe44002f30 tx=0x7efe44007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:36.128 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.121+0000 7efe4d1d8700 1 -- 172.21.15.170:0/1560719772 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7efe44011870 con 0x7efe48125060 2020-03-19T21:18:36.129 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.121+0000 7efe5043e700 1 -- 172.21.15.170:0/1560719772 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe48125060 msgr2=0x7efe48127450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:36.129 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.121+0000 7efe5043e700 1 --2- 172.21.15.170:0/1560719772 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe48125060 0x7efe48127450 secure :-1 s=READY pgs=279 cs=0 l=1 rx=0x7efe44002f30 tx=0x7efe44007a40).stop 2020-03-19T21:18:36.129 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.121+0000 7efe5043e700 1 -- 172.21.15.170:0/1560719772 shutdown_connections 2020-03-19T21:18:36.129 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.121+0000 7efe5043e700 1 --2- 172.21.15.170:0/1560719772 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe48125060 0x7efe48127450 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:36.129 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.121+0000 7efe5043e700 1 -- 172.21.15.170:0/1560719772 >> 172.21.15.170:0/1560719772 conn(0x7efe48091390 msgr2=0x7efe48091770 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:36.129 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.121+0000 7efe5043e700 1 -- 172.21.15.170:0/1560719772 shutdown_connections 2020-03-19T21:18:36.129 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.121+0000 7efe5043e700 1 -- 172.21.15.170:0/1560719772 wait complete. 2020-03-19T21:18:36.130 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.122+0000 7efe5043e700 1 Processor -- start 2020-03-19T21:18:36.130 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.122+0000 7efe5043e700 1 -- start start 2020-03-19T21:18:36.130 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.122+0000 7efe5043e700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7efe48065970 0x7efe48132980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:36.130 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.122+0000 7efe5043e700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe48113730 con 0x7efe48125060 2020-03-19T21:18:36.130 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.122+0000 7efe5043e700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe48114f50 con 0x7efe48065970 2020-03-19T21:18:36.130 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.122+0000 7efe4e1da700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7efe48065970 0x7efe48132980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:36.130 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.122+0000 7efe4d9d9700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7efe48125060 0x7efe48132de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:54100/0 (socket says 172.21.15.170:54100) 2020-03-19T21:18:36.131 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.122+0000 7efe4d9d9700 1 -- 172.21.15.170:0/668902634 learned_addr learned my addr 172.21.15.170:0/668902634 (peer_addr_for_me v1:172.21.15.170:0/0) 2020-03-19T21:18:36.131 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.123+0000 7efe4e1da700 1 -- 172.21.15.170:0/668902634 >> v1:172.21.15.170:6789/0 conn(0x7efe48125060 legacy=0x7efe48132de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:36.131 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.123+0000 7efe377fe700 1 -- 172.21.15.170:0/668902634 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2675436513 0 0) 0x7efe48113730 con 0x7efe48125060 2020-03-19T21:18:36.131 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.123+0000 7efe4e1da700 1 -- 172.21.15.170:0/668902634 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe481134d0 con 0x7efe48065970 2020-03-19T21:18:36.131 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.123+0000 7efe4e1da700 1 --2- 172.21.15.170:0/668902634 >> v2:172.21.15.170:3300/0 conn(0x7efe48065970 0x7efe48132980 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7efe48113c90 tx=0x7efe380108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:36.131 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.123+0000 7efe4e1da700 1 --2- 172.21.15.170:0/668902634 >> v2:172.21.15.170:3300/0 conn(0x7efe48065970 0x7efe48132980 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7efe48113c90 tx=0x7efe380108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:36.131 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.123+0000 7efe4e1da700 1 --2- 172.21.15.170:0/668902634 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe48065970 0x7efe48132980 secure :-1 s=READY pgs=281 cs=0 l=1 rx=0x7efe48113c90 tx=0x7efe380108a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:36.131 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.123+0000 7efe4e1da700 1 --2- 172.21.15.170:0/668902634 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe48065970 0x7efe48132980 secure :-1 s=THROTTLE_DONE pgs=281 cs=0 l=1 rx=0x7efe48113c90 tx=0x7efe380108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:36.132 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.123+0000 7efe377fe700 1 -- 172.21.15.170:0/668902634 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7efe38005b20 con 0x7efe48065970 2020-03-19T21:18:36.132 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.123+0000 7efe4e1da700 1 --2- 172.21.15.170:0/668902634 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe48065970 0x7efe48132980 secure :-1 s=THROTTLE_DONE pgs=281 cs=0 l=1 rx=0x7efe48113c90 tx=0x7efe380108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:36.132 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.123+0000 7efe4e1da700 1 --2- 172.21.15.170:0/668902634 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe48065970 0x7efe48132980 secure :-1 s=THROTTLE_DONE pgs=281 cs=0 l=1 rx=0x7efe48113c90 tx=0x7efe380108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:36.132 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.123+0000 7efe377fe700 1 -- 172.21.15.170:0/668902634 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe38005c80 con 0x7efe48065970 2020-03-19T21:18:36.132 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.123+0000 7efe377fe700 1 -- 172.21.15.170:0/668902634 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7efe380060c0 con 0x7efe48065970 2020-03-19T21:18:36.132 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.123+0000 7efe5043e700 1 -- 172.21.15.170:0/668902634 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe481130c0 con 0x7efe48065970 2020-03-19T21:18:36.132 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.123+0000 7efe5043e700 1 -- 172.21.15.170:0/668902634 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe481334f0 con 0x7efe48065970 2020-03-19T21:18:36.133 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.123+0000 7efe4e1da700 1 --2- 172.21.15.170:0/668902634 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe48065970 0x7efe48132980 secure :-1 s=THROTTLE_DONE pgs=281 cs=0 l=1 rx=0x7efe48113c90 tx=0x7efe380108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:36.133 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.124+0000 7efe377fe700 1 -- 172.21.15.170:0/668902634 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 4) v1 ==== 29603+0+0 (secure 0 0 0) 0x7efe3800a320 con 0x7efe48065970 2020-03-19T21:18:36.133 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.124+0000 7efe4e1da700 1 --2- 172.21.15.170:0/668902634 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe48065970 0x7efe48132980 secure :-1 s=THROTTLE_DONE pgs=281 cs=0 l=1 rx=0x7efe48113c90 tx=0x7efe380108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:36.133 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.124+0000 7efe377fe700 1 --2- 172.21.15.170:0/668902634 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7efe28028280 0x7efe2802a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:36.133 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.124+0000 7efe377fe700 1 -- 172.21.15.170:0/668902634 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 3786+0+0 (secure 0 0 0) 0x7efe38033f20 con 0x7efe48065970 2020-03-19T21:18:36.133 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.124+0000 7efe4d9d9700 1 --2- 172.21.15.170:0/668902634 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7efe28028280 0x7efe2802a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:36.133 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.124+0000 7efe5043e700 1 --2- 172.21.15.170:0/668902634 >> [v2:172.21.15.170:6808/20681,v1:172.21.15.170:6809/20681] conn(0x7efe48055b30 0x7efe48055f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:36.134 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.124+0000 7efe4d9d9700 1 --2- 172.21.15.170:0/668902634 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7efe28028280 0x7efe2802a740 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7efe440101c0 tx=0x7efe44007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:36.134 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.124+0000 7efe5043e700 1 -- 172.21.15.170:0/668902634 --> [v2:172.21.15.170:6808/20681,v1:172.21.15.170:6809/20681] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7efe48056db0 con 0x7efe48055b30 2020-03-19T21:18:36.134 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.124+0000 7efe4d9d9700 1 --2- 172.21.15.170:0/668902634 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7efe28028280 0x7efe2802a740 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7efe440101c0 tx=0x7efe44007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:36.134 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.124+0000 7efe4d9d9700 1 --2- 172.21.15.170:0/668902634 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7efe28028280 0x7efe2802a740 secure :-1 s=READY pgs=69 cs=0 l=1 rx=0x7efe440101c0 tx=0x7efe44007a40).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:36.134 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.124+0000 7efe4e9db700 1 --2- 172.21.15.170:0/668902634 >> [v2:172.21.15.170:6808/20681,v1:172.21.15.170:6809/20681] conn(0x7efe48055b30 0x7efe48055f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:36.134 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.124+0000 7efe4e9db700 1 --2- 172.21.15.170:0/668902634 >> [v2:172.21.15.170:6808/20681,v1:172.21.15.170:6809/20681] conn(0x7efe48055b30 0x7efe48055f10 crc :-1 s=READY pgs=8 cs=0 l=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:36.134 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.125+0000 7efe377fe700 1 -- 172.21.15.170:0/668902634 <== osd.1 v2:172.21.15.170:6808/20681 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23288 (crc 0 0 0) 0x7efe48056db0 con 0x7efe48055b30 2020-03-19T21:18:36.142 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.141+0000 7efe5043e700 1 -- 172.21.15.170:0/668902634 --> [v2:172.21.15.170:6808/20681,v1:172.21.15.170:6809/20681] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7efe48067160 con 0x7efe48055b30 2020-03-19T21:18:36.143 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.141+0000 7efe377fe700 1 -- 172.21.15.170:0/668902634 <== osd.1 v2:172.21.15.170:6808/20681 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7efe48067160 con 0x7efe48055b30 2020-03-19T21:18:36.144 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.142+0000 7efe5043e700 1 -- 172.21.15.170:0/668902634 >> [v2:172.21.15.170:6808/20681,v1:172.21.15.170:6809/20681] conn(0x7efe48055b30 msgr2=0x7efe48055f10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:36.144 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.142+0000 7efe5043e700 1 --2- 172.21.15.170:0/668902634 >> [v2:172.21.15.170:6808/20681,v1:172.21.15.170:6809/20681] conn(0x7efe48055b30 0x7efe48055f10 crc :-1 s=READY pgs=8 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:36.144 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.142+0000 7efe5043e700 1 -- 172.21.15.170:0/668902634 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7efe28028280 msgr2=0x7efe2802a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:36.144 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.142+0000 7efe5043e700 1 --2- 172.21.15.170:0/668902634 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7efe28028280 0x7efe2802a740 secure :-1 s=READY pgs=69 cs=0 l=1 rx=0x7efe440101c0 tx=0x7efe44007a40).stop 2020-03-19T21:18:36.144 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.142+0000 7efe5043e700 1 -- 172.21.15.170:0/668902634 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe48065970 msgr2=0x7efe48132980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:36.144 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.142+0000 7efe5043e700 1 --2- 172.21.15.170:0/668902634 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe48065970 0x7efe48132980 secure :-1 s=READY pgs=281 cs=0 l=1 rx=0x7efe48113c90 tx=0x7efe380108a0).stop 2020-03-19T21:18:36.145 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.142+0000 7efe5043e700 1 -- 172.21.15.170:0/668902634 shutdown_connections 2020-03-19T21:18:36.145 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.142+0000 7efe5043e700 1 --2- 172.21.15.170:0/668902634 >> [v2:172.21.15.170:6808/20681,v1:172.21.15.170:6809/20681] conn(0x7efe48055b30 0x7efe48055f10 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:36.145 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.142+0000 7efe5043e700 1 --2- 172.21.15.170:0/668902634 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7efe28028280 0x7efe2802a740 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:36.145 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.142+0000 7efe5043e700 1 --2- 172.21.15.170:0/668902634 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe48065970 0x7efe48132980 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:36.145 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.142+0000 7efe5043e700 1 -- 172.21.15.170:0/668902634 >> 172.21.15.170:0/668902634 conn(0x7efe48091390 msgr2=0x7efe48124940 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:36.145 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.142+0000 7efe5043e700 1 -- 172.21.15.170:0/668902634 shutdown_connections 2020-03-19T21:18:36.148 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.142+0000 7efe5043e700 1 -- 172.21.15.170:0/668902634 wait complete. 2020-03-19T21:18:36.148 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2020-03-19T21:18:36.150 INFO:teuthology.orchestra.run.smithi170.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2020-03-19T21:18:36.150 INFO:teuthology.orchestra.run.smithi170.stdout:} 2020-03-19T21:18:36.153 INFO:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd deep-scrub 1 2020-03-19T21:18:36.259 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.258+0000 7f37b3b96700 1 Processor -- start 2020-03-19T21:18:36.259 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.258+0000 7f37b3b96700 1 -- start start 2020-03-19T21:18:36.261 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.259+0000 7f37b3b96700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f37ac11fd60 0x7f37ac127a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T21:18:36.262 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.259+0000 7f37b3b96700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37ac0911c0 con 0x7f37ac091eb0 2020-03-19T21:18:36.262 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.259+0000 7f37b3b96700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37ac067130 con 0x7f37ac11fd60 2020-03-19T21:18:36.262 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.259+0000 7f37b1131700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f37ac11fd60 0x7f37ac127a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:36.262 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.259+0000 7f37b1932700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f37ac091eb0 0x7f37ac11f650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:54106/0 (socket says 172.21.15.170:54106) 2020-03-19T21:18:36.262 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.259+0000 7f37b1131700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f37ac11fd60 0x7f37ac127a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53344/0 (socket says 172.21.15.170:53344) 2020-03-19T21:18:36.262 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.259+0000 7f37b1932700 1 -- 172.21.15.170:0/597339792 learned_addr learned my addr 172.21.15.170:0/597339792 (peer_addr_for_me v1:172.21.15.170:0/0) 2020-03-19T21:18:36.262 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.259+0000 7f37b1131700 1 -- 172.21.15.170:0/597339792 >> v1:172.21.15.170:6789/0 conn(0x7f37ac091eb0 legacy=0x7f37ac11f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:36.263 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.259+0000 7f37b1131700 1 -- 172.21.15.170:0/597339792 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37ac11f340 con 0x7f37ac11fd60 2020-03-19T21:18:36.263 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.259+0000 7f37b1131700 1 --2- 172.21.15.170:0/597339792 >> v2:172.21.15.170:3300/0 conn(0x7f37ac11fd60 0x7f37ac127a70 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f379c002f30 tx=0x7f379c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:36.263 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.259+0000 7f37b1131700 1 --2- 172.21.15.170:0/597339792 >> v2:172.21.15.170:3300/0 conn(0x7f37ac11fd60 0x7f37ac127a70 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f379c002f30 tx=0x7f379c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:36.263 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.259+0000 7f37b1131700 1 --2- 172.21.15.170:0/597339792 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f37ac11fd60 0x7f37ac127a70 secure :-1 s=READY pgs=283 cs=0 l=1 rx=0x7f379c002f30 tx=0x7f379c007a40).ready entity=mon.0 client_cookie=d2aea86d46b020c0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:36.263 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.259+0000 7f37b1131700 1 --2- 172.21.15.170:0/597339792 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f37ac11fd60 0x7f37ac127a70 secure :-1 s=THROTTLE_DONE pgs=283 cs=0 l=1 rx=0x7f379c002f30 tx=0x7f379c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:36.263 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.259+0000 7f37b1131700 1 --2- 172.21.15.170:0/597339792 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f37ac11fd60 0x7f37ac127a70 secure :-1 s=THROTTLE_DONE pgs=283 cs=0 l=1 rx=0x7f379c002f30 tx=0x7f379c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:36.264 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.259+0000 7f37b0930700 1 -- 172.21.15.170:0/597339792 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f379c008e80 con 0x7f37ac11fd60 2020-03-19T21:18:36.264 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.259+0000 7f37b1131700 1 --2- 172.21.15.170:0/597339792 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f37ac11fd60 0x7f37ac127a70 secure :-1 s=THROTTLE_DONE pgs=283 cs=0 l=1 rx=0x7f379c002f30 tx=0x7f379c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:36.264 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.259+0000 7f37b0930700 1 -- 172.21.15.170:0/597339792 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f379c0113f0 con 0x7f37ac11fd60 2020-03-19T21:18:36.264 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.259+0000 7f37b0930700 1 -- 172.21.15.170:0/597339792 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f379c0116d0 con 0x7f37ac11fd60 2020-03-19T21:18:36.264 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.259+0000 7f37b3b96700 1 -- 172.21.15.170:0/597339792 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f37ac11fd60 msgr2=0x7f37ac127a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:36.264 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.259+0000 7f37b3b96700 1 --2- 172.21.15.170:0/597339792 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f37ac11fd60 0x7f37ac127a70 secure :-1 s=READY pgs=283 cs=0 l=1 rx=0x7f379c002f30 tx=0x7f379c007a40).stop 2020-03-19T21:18:36.264 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.260+0000 7f37b3b96700 1 -- 172.21.15.170:0/597339792 shutdown_connections 2020-03-19T21:18:36.265 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.260+0000 7f37b3b96700 1 --2- 172.21.15.170:0/597339792 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f37ac11fd60 0x7f37ac127a70 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:36.265 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.260+0000 7f37b3b96700 1 -- 172.21.15.170:0/597339792 >> 172.21.15.170:0/597339792 conn(0x7f37ac091390 msgr2=0x7f37ac0917a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:36.265 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.260+0000 7f37b3b96700 1 -- 172.21.15.170:0/597339792 shutdown_connections 2020-03-19T21:18:36.265 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.260+0000 7f37b3b96700 1 -- 172.21.15.170:0/597339792 wait complete. 2020-03-19T21:18:36.268 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.261+0000 7f37b3b96700 1 Processor -- start 2020-03-19T21:18:36.269 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.261+0000 7f37b3b96700 1 -- start start 2020-03-19T21:18:36.269 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.261+0000 7f37b3b96700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f37ac091eb0 0x7f37ac12ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:36.269 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.261+0000 7f37b3b96700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37ac064e00 con 0x7f37ac11fd60 2020-03-19T21:18:36.269 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.261+0000 7f37b3b96700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37ac063700 con 0x7f37ac091eb0 2020-03-19T21:18:36.269 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.261+0000 7f37b1932700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f37ac091eb0 0x7f37ac12ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:36.269 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.261+0000 7f37b1131700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f37ac11fd60 0x7f37ac12f0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:54112/0 (socket says 172.21.15.170:54112) 2020-03-19T21:18:36.270 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.261+0000 7f37b1131700 1 -- 172.21.15.170:0/892493403 learned_addr learned my addr 172.21.15.170:0/892493403 (peer_addr_for_me v1:172.21.15.170:0/0) 2020-03-19T21:18:36.270 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.262+0000 7f37b1932700 1 -- 172.21.15.170:0/892493403 >> v1:172.21.15.170:6789/0 conn(0x7f37ac11fd60 legacy=0x7f37ac12f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:36.270 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.262+0000 7f37b1932700 1 -- 172.21.15.170:0/892493403 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37ac066410 con 0x7f37ac091eb0 2020-03-19T21:18:36.270 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.262+0000 7f379affd700 1 -- 172.21.15.170:0/892493403 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1729713134 0 0) 0x7f37ac064e00 con 0x7f37ac11fd60 2020-03-19T21:18:36.270 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.262+0000 7f37b1932700 1 --2- 172.21.15.170:0/892493403 >> v2:172.21.15.170:3300/0 conn(0x7f37ac091eb0 0x7f37ac12ec70 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f37ac067970 tx=0x7f37a80108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:36.270 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.262+0000 7f37b1932700 1 --2- 172.21.15.170:0/892493403 >> v2:172.21.15.170:3300/0 conn(0x7f37ac091eb0 0x7f37ac12ec70 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f37ac067970 tx=0x7f37a80108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:36.270 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.262+0000 7f37b1932700 1 --2- 172.21.15.170:0/892493403 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f37ac091eb0 0x7f37ac12ec70 secure :-1 s=READY pgs=285 cs=0 l=1 rx=0x7f37ac067970 tx=0x7f37a80108a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:36.271 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.262+0000 7f37b1932700 1 --2- 172.21.15.170:0/892493403 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f37ac091eb0 0x7f37ac12ec70 secure :-1 s=THROTTLE_DONE pgs=285 cs=0 l=1 rx=0x7f37ac067970 tx=0x7f37a80108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:36.271 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.262+0000 7f379affd700 1 -- 172.21.15.170:0/892493403 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f37a8005b20 con 0x7f37ac091eb0 2020-03-19T21:18:36.271 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.262+0000 7f37b1932700 1 --2- 172.21.15.170:0/892493403 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f37ac091eb0 0x7f37ac12ec70 secure :-1 s=THROTTLE_DONE pgs=285 cs=0 l=1 rx=0x7f37ac067970 tx=0x7f37a80108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:36.271 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.262+0000 7f37b1932700 1 --2- 172.21.15.170:0/892493403 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f37ac091eb0 0x7f37ac12ec70 secure :-1 s=THROTTLE_DONE pgs=285 cs=0 l=1 rx=0x7f37ac067970 tx=0x7f37a80108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:36.271 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.262+0000 7f379affd700 1 -- 172.21.15.170:0/892493403 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37a8005c80 con 0x7f37ac091eb0 2020-03-19T21:18:36.271 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.262+0000 7f37b3b96700 1 -- 172.21.15.170:0/892493403 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37ac1239c0 con 0x7f37ac091eb0 2020-03-19T21:18:36.271 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.262+0000 7f379affd700 1 -- 172.21.15.170:0/892493403 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f37a80060c0 con 0x7f37ac091eb0 2020-03-19T21:18:36.272 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.262+0000 7f37b3b96700 1 -- 172.21.15.170:0/892493403 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37ac065720 con 0x7f37ac091eb0 2020-03-19T21:18:36.272 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.262+0000 7f37b1932700 1 --2- 172.21.15.170:0/892493403 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f37ac091eb0 0x7f37ac12ec70 secure :-1 s=THROTTLE_DONE pgs=285 cs=0 l=1 rx=0x7f37ac067970 tx=0x7f37a80108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:36.272 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.262+0000 7f379affd700 1 -- 172.21.15.170:0/892493403 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 4) v1 ==== 29603+0+0 (secure 0 0 0) 0x7f37a800a320 con 0x7f37ac091eb0 2020-03-19T21:18:36.272 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.262+0000 7f37b1932700 1 --2- 172.21.15.170:0/892493403 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f37ac091eb0 0x7f37ac12ec70 secure :-1 s=THROTTLE_DONE pgs=285 cs=0 l=1 rx=0x7f37ac067970 tx=0x7f37a80108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:36.272 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.263+0000 7f379affd700 1 --2- 172.21.15.170:0/892493403 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f378c028280 0x7f378c02a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:36.272 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.263+0000 7f379affd700 1 -- 172.21.15.170:0/892493403 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 3786+0+0 (secure 0 0 0) 0x7f37a8034010 con 0x7f37ac091eb0 2020-03-19T21:18:36.272 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.263+0000 7f37b1131700 1 --2- 172.21.15.170:0/892493403 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f378c028280 0x7f378c02a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:36.273 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.263+0000 7f37b1131700 1 --2- 172.21.15.170:0/892493403 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f378c028280 0x7f378c02a740 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f379c0101c0 tx=0x7f379c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:36.273 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.263+0000 7f37b1131700 1 --2- 172.21.15.170:0/892493403 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f378c028280 0x7f378c02a740 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f379c0101c0 tx=0x7f379c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:36.273 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.263+0000 7f37b1131700 1 --2- 172.21.15.170:0/892493403 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f378c028280 0x7f378c02a740 secure :-1 s=READY pgs=70 cs=0 l=1 rx=0x7f379c0101c0 tx=0x7f379c007a40).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:36.273 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.263+0000 7f37b3b96700 1 -- 172.21.15.170:0/892493403 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37ac056db0 con 0x7f37ac091eb0 2020-03-19T21:18:36.273 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.267+0000 7f37b1932700 1 --2- 172.21.15.170:0/892493403 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f37ac091eb0 0x7f37ac12ec70 secure :-1 s=THROTTLE_DONE pgs=285 cs=0 l=1 rx=0x7f37ac067970 tx=0x7f37a80108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:36.273 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.267+0000 7f379affd700 1 -- 172.21.15.170:0/892493403 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+126772 (secure 0 0 0) 0x7f37a8006220 con 0x7f37ac091eb0 2020-03-19T21:18:36.407 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.406+0000 7f37b3b96700 1 -- 172.21.15.170:0/892493403 --> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f37ac0693f0 con 0x7f378c028280 2020-03-19T21:18:36.407 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.406+0000 7f37b1131700 1 --2- 172.21.15.170:0/892493403 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f378c028280 0x7f378c02a740 secure :-1 s=THROTTLE_DONE pgs=70 cs=0 l=1 rx=0x7f379c0101c0 tx=0x7f379c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:36.407 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.406+0000 7f379affd700 1 -- 172.21.15.170:0/892493403 <== mgr.4102 v2:172.21.15.170:6824/20154 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f37ac0693f0 con 0x7f378c028280 2020-03-19T21:18:36.409 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.407+0000 7f37b3b96700 1 -- 172.21.15.170:0/892493403 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f378c028280 msgr2=0x7f378c02a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:36.409 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.407+0000 7f37b3b96700 1 --2- 172.21.15.170:0/892493403 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f378c028280 0x7f378c02a740 secure :-1 s=READY pgs=70 cs=0 l=1 rx=0x7f379c0101c0 tx=0x7f379c007a40).stop 2020-03-19T21:18:36.409 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.407+0000 7f37b3b96700 1 -- 172.21.15.170:0/892493403 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f37ac091eb0 msgr2=0x7f37ac12ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:36.409 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.407+0000 7f37b3b96700 1 --2- 172.21.15.170:0/892493403 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f37ac091eb0 0x7f37ac12ec70 secure :-1 s=READY pgs=285 cs=0 l=1 rx=0x7f37ac067970 tx=0x7f37a80108a0).stop 2020-03-19T21:18:36.410 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.407+0000 7f37b3b96700 1 -- 172.21.15.170:0/892493403 shutdown_connections 2020-03-19T21:18:36.410 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.407+0000 7f37b3b96700 1 --2- 172.21.15.170:0/892493403 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f378c028280 0x7f378c02a740 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:36.410 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.407+0000 7f37b3b96700 1 --2- 172.21.15.170:0/892493403 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f37ac091eb0 0x7f37ac12ec70 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:36.410 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.407+0000 7f37b3b96700 1 -- 172.21.15.170:0/892493403 >> 172.21.15.170:0/892493403 conn(0x7f37ac091390 msgr2=0x7f37ac093c30 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:36.410 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.407+0000 7f37b3b96700 1 -- 172.21.15.170:0/892493403 shutdown_connections 2020-03-19T21:18:36.410 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.408+0000 7f37b3b96700 1 -- 172.21.15.170:0/892493403 wait complete. 2020-03-19T21:18:36.411 INFO:teuthology.orchestra.run.smithi170.stderr:instructed osd(s) 1 to deep-scrub 2020-03-19T21:18:36.426 INFO:tasks.ceph:Scrubbing osd.2 2020-03-19T21:18:36.426 INFO:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early tell osd.2 config set osd_debug_deep_scrub_sleep 0 2020-03-19T21:18:36.535 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.534+0000 7fb313a47700 1 Processor -- start 2020-03-19T21:18:36.536 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.534+0000 7fb313a47700 1 -- start start 2020-03-19T21:18:36.538 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.535+0000 7fb313a47700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fb30c125060 0x7fb30c127450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T21:18:36.538 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.535+0000 7fb313a47700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb30c060f10 con 0x7fb30c065970 2020-03-19T21:18:36.538 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.535+0000 7fb313a47700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb30c061490 con 0x7fb30c125060 2020-03-19T21:18:36.538 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.535+0000 7fb310fe2700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fb30c125060 0x7fb30c127450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:36.539 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.535+0000 7fb3117e3700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fb30c065970 0x7fb30c065d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:54116/0 (socket says 172.21.15.170:54116) 2020-03-19T21:18:36.539 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.535+0000 7fb3117e3700 1 -- 172.21.15.170:0/2160669872 learned_addr learned my addr 172.21.15.170:0/2160669872 (peer_addr_for_me v1:172.21.15.170:0/0) 2020-03-19T21:18:36.539 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.535+0000 7fb310fe2700 1 -- 172.21.15.170:0/2160669872 >> v1:172.21.15.170:6789/0 conn(0x7fb30c065970 legacy=0x7fb30c065d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:36.539 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.535+0000 7fb310fe2700 1 -- 172.21.15.170:0/2160669872 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb30c061730 con 0x7fb30c125060 2020-03-19T21:18:36.539 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.535+0000 7fb2fbfff700 1 -- 172.21.15.170:0/2160669872 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 978149527 0 0) 0x7fb2fc0010d0 con 0x7fb30c065970 2020-03-19T21:18:36.539 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.535+0000 7fb310fe2700 1 --2- 172.21.15.170:0/2160669872 >> v2:172.21.15.170:3300/0 conn(0x7fb30c125060 0x7fb30c127450 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7fb308002f30 tx=0x7fb308007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:36.539 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.535+0000 7fb310fe2700 1 --2- 172.21.15.170:0/2160669872 >> v2:172.21.15.170:3300/0 conn(0x7fb30c125060 0x7fb30c127450 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7fb308002f30 tx=0x7fb308007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:36.540 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.535+0000 7fb310fe2700 1 --2- 172.21.15.170:0/2160669872 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb30c125060 0x7fb30c127450 secure :-1 s=READY pgs=287 cs=0 l=1 rx=0x7fb308002f30 tx=0x7fb308007a40).ready entity=mon.0 client_cookie=557e277f44af07a6 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:36.540 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.536+0000 7fb310fe2700 1 --2- 172.21.15.170:0/2160669872 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb30c125060 0x7fb30c127450 secure :-1 s=THROTTLE_DONE pgs=287 cs=0 l=1 rx=0x7fb308002f30 tx=0x7fb308007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:36.540 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.536+0000 7fb2fbfff700 1 -- 172.21.15.170:0/2160669872 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fb30800ea30 con 0x7fb30c125060 2020-03-19T21:18:36.540 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.536+0000 7fb310fe2700 1 --2- 172.21.15.170:0/2160669872 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb30c125060 0x7fb30c127450 secure :-1 s=THROTTLE_DONE pgs=287 cs=0 l=1 rx=0x7fb308002f30 tx=0x7fb308007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:36.540 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.536+0000 7fb2fbfff700 1 -- 172.21.15.170:0/2160669872 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb30800eb90 con 0x7fb30c125060 2020-03-19T21:18:36.540 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.536+0000 7fb310fe2700 1 --2- 172.21.15.170:0/2160669872 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb30c125060 0x7fb30c127450 secure :-1 s=THROTTLE_DONE pgs=287 cs=0 l=1 rx=0x7fb308002f30 tx=0x7fb308007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:36.540 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.536+0000 7fb2fbfff700 1 -- 172.21.15.170:0/2160669872 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fb30800ee30 con 0x7fb30c125060 2020-03-19T21:18:36.541 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.536+0000 7fb313a47700 1 -- 172.21.15.170:0/2160669872 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb30c125060 msgr2=0x7fb30c127450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:36.541 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.536+0000 7fb313a47700 1 --2- 172.21.15.170:0/2160669872 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb30c125060 0x7fb30c127450 secure :-1 s=READY pgs=287 cs=0 l=1 rx=0x7fb308002f30 tx=0x7fb308007a40).stop 2020-03-19T21:18:36.541 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.536+0000 7fb313a47700 1 -- 172.21.15.170:0/2160669872 shutdown_connections 2020-03-19T21:18:36.541 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.536+0000 7fb313a47700 1 --2- 172.21.15.170:0/2160669872 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb30c125060 0x7fb30c127450 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:36.541 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.536+0000 7fb313a47700 1 -- 172.21.15.170:0/2160669872 >> 172.21.15.170:0/2160669872 conn(0x7fb30c091390 msgr2=0x7fb30c091770 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:36.541 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.536+0000 7fb313a47700 1 -- 172.21.15.170:0/2160669872 shutdown_connections 2020-03-19T21:18:36.541 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.536+0000 7fb313a47700 1 -- 172.21.15.170:0/2160669872 wait complete. 2020-03-19T21:18:36.542 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.537+0000 7fb313a47700 1 Processor -- start 2020-03-19T21:18:36.542 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.537+0000 7fb313a47700 1 -- start start 2020-03-19T21:18:36.545 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.537+0000 7fb313a47700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fb30c125060 0x7fb30c12f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:36.545 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.537+0000 7fb313a47700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb30c10da70 con 0x7fb30c065970 2020-03-19T21:18:36.545 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.537+0000 7fb313a47700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb30c1115e0 con 0x7fb30c125060 2020-03-19T21:18:36.545 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.537+0000 7fb310fe2700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fb30c125060 0x7fb30c12f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:36.545 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.537+0000 7fb3117e3700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fb30c065970 0x7fb30c12ebb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:54120/0 (socket says 172.21.15.170:54120) 2020-03-19T21:18:36.545 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.537+0000 7fb3117e3700 1 -- 172.21.15.170:0/549818682 learned_addr learned my addr 172.21.15.170:0/549818682 (peer_addr_for_me v1:172.21.15.170:0/0) 2020-03-19T21:18:36.546 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.538+0000 7fb310fe2700 1 -- 172.21.15.170:0/549818682 >> v1:172.21.15.170:6789/0 conn(0x7fb30c065970 legacy=0x7fb30c12ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:36.546 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.538+0000 7fb2fa7fc700 1 -- 172.21.15.170:0/549818682 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4048125331 0 0) 0x7fb30c10da70 con 0x7fb30c065970 2020-03-19T21:18:36.546 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.538+0000 7fb310fe2700 1 -- 172.21.15.170:0/549818682 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb30c12c990 con 0x7fb30c125060 2020-03-19T21:18:36.546 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.538+0000 7fb310fe2700 1 --2- 172.21.15.170:0/549818682 >> v2:172.21.15.170:3300/0 conn(0x7fb30c125060 0x7fb30c12f2c0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7fb308002f30 tx=0x7fb308007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:36.546 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.538+0000 7fb310fe2700 1 --2- 172.21.15.170:0/549818682 >> v2:172.21.15.170:3300/0 conn(0x7fb30c125060 0x7fb30c12f2c0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7fb308002f30 tx=0x7fb308007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:36.546 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.538+0000 7fb310fe2700 1 --2- 172.21.15.170:0/549818682 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb30c125060 0x7fb30c12f2c0 secure :-1 s=READY pgs=289 cs=0 l=1 rx=0x7fb308002f30 tx=0x7fb308007a40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:36.546 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.538+0000 7fb310fe2700 1 --2- 172.21.15.170:0/549818682 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb30c125060 0x7fb30c12f2c0 secure :-1 s=THROTTLE_DONE pgs=289 cs=0 l=1 rx=0x7fb308002f30 tx=0x7fb308007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:36.547 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.538+0000 7fb2fa7fc700 1 -- 172.21.15.170:0/549818682 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fb308007290 con 0x7fb30c125060 2020-03-19T21:18:36.547 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.538+0000 7fb310fe2700 1 --2- 172.21.15.170:0/549818682 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb30c125060 0x7fb30c12f2c0 secure :-1 s=THROTTLE_DONE pgs=289 cs=0 l=1 rx=0x7fb308002f30 tx=0x7fb308007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:36.547 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.538+0000 7fb310fe2700 1 --2- 172.21.15.170:0/549818682 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb30c125060 0x7fb30c12f2c0 secure :-1 s=THROTTLE_DONE pgs=289 cs=0 l=1 rx=0x7fb308002f30 tx=0x7fb308007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:36.547 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.538+0000 7fb2fa7fc700 1 -- 172.21.15.170:0/549818682 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3080073f0 con 0x7fb30c125060 2020-03-19T21:18:36.547 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.538+0000 7fb313a47700 1 -- 172.21.15.170:0/549818682 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb30c10f250 con 0x7fb30c125060 2020-03-19T21:18:36.547 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.538+0000 7fb2fa7fc700 1 -- 172.21.15.170:0/549818682 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fb3080058d0 con 0x7fb30c125060 2020-03-19T21:18:36.548 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.538+0000 7fb313a47700 1 -- 172.21.15.170:0/549818682 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb30c10fe90 con 0x7fb30c125060 2020-03-19T21:18:36.548 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.538+0000 7fb310fe2700 1 --2- 172.21.15.170:0/549818682 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb30c125060 0x7fb30c12f2c0 secure :-1 s=THROTTLE_DONE pgs=289 cs=0 l=1 rx=0x7fb308002f30 tx=0x7fb308007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:36.548 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.539+0000 7fb2fa7fc700 1 -- 172.21.15.170:0/549818682 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 4) v1 ==== 29603+0+0 (secure 0 0 0) 0x7fb308005ad0 con 0x7fb30c125060 2020-03-19T21:18:36.548 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.539+0000 7fb310fe2700 1 --2- 172.21.15.170:0/549818682 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb30c125060 0x7fb30c12f2c0 secure :-1 s=THROTTLE_DONE pgs=289 cs=0 l=1 rx=0x7fb308002f30 tx=0x7fb308007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:36.548 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.539+0000 7fb2fa7fc700 1 --2- 172.21.15.170:0/549818682 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7fb2ec028200 0x7fb2ec02a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:36.548 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.539+0000 7fb2fa7fc700 1 -- 172.21.15.170:0/549818682 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 3786+0+0 (secure 0 0 0) 0x7fb308038e30 con 0x7fb30c125060 2020-03-19T21:18:36.548 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.539+0000 7fb3117e3700 1 --2- 172.21.15.170:0/549818682 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7fb2ec028200 0x7fb2ec02a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:36.549 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.539+0000 7fb313a47700 1 --2- 172.21.15.170:0/549818682 >> [v2:172.21.15.170:6816/20682,v1:172.21.15.170:6817/20682] conn(0x7fb30c056040 0x7fb30c056420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:36.549 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.539+0000 7fb313a47700 1 -- 172.21.15.170:0/549818682 --> [v2:172.21.15.170:6816/20682,v1:172.21.15.170:6817/20682] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb30c094660 con 0x7fb30c056040 2020-03-19T21:18:36.549 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.539+0000 7fb3117e3700 1 --2- 172.21.15.170:0/549818682 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7fb2ec028200 0x7fb2ec02a6c0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7fb2fc005500 tx=0x7fb2fc0016e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:36.549 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.539+0000 7fb3117e3700 1 --2- 172.21.15.170:0/549818682 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7fb2ec028200 0x7fb2ec02a6c0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7fb2fc005500 tx=0x7fb2fc0016e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:36.549 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.539+0000 7fb3117e3700 1 --2- 172.21.15.170:0/549818682 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7fb2ec028200 0x7fb2ec02a6c0 secure :-1 s=READY pgs=71 cs=0 l=1 rx=0x7fb2fc005500 tx=0x7fb2fc0016e0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:36.549 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.539+0000 7fb311fe4700 1 --2- 172.21.15.170:0/549818682 >> [v2:172.21.15.170:6816/20682,v1:172.21.15.170:6817/20682] conn(0x7fb30c056040 0x7fb30c056420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:36.549 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.539+0000 7fb311fe4700 1 --2- 172.21.15.170:0/549818682 >> [v2:172.21.15.170:6816/20682,v1:172.21.15.170:6817/20682] conn(0x7fb30c056040 0x7fb30c056420 crc :-1 s=READY pgs=8 cs=0 l=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:36.550 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.540+0000 7fb2fa7fc700 1 -- 172.21.15.170:0/549818682 <== osd.2 v2:172.21.15.170:6816/20682 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23288 (crc 0 0 0) 0x7fb30c094660 con 0x7fb30c056040 2020-03-19T21:18:36.557 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.556+0000 7fb313a47700 1 -- 172.21.15.170:0/549818682 --> [v2:172.21.15.170:6816/20682,v1:172.21.15.170:6817/20682] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fb30c056b20 con 0x7fb30c056040 2020-03-19T21:18:36.559 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.557+0000 7fb2fa7fc700 1 -- 172.21.15.170:0/549818682 <== osd.2 v2:172.21.15.170:6816/20682 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fb30c056b20 con 0x7fb30c056040 2020-03-19T21:18:36.559 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.557+0000 7fb313a47700 1 -- 172.21.15.170:0/549818682 >> [v2:172.21.15.170:6816/20682,v1:172.21.15.170:6817/20682] conn(0x7fb30c056040 msgr2=0x7fb30c056420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:36.559 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.557+0000 7fb313a47700 1 --2- 172.21.15.170:0/549818682 >> [v2:172.21.15.170:6816/20682,v1:172.21.15.170:6817/20682] conn(0x7fb30c056040 0x7fb30c056420 crc :-1 s=READY pgs=8 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:36.559 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.557+0000 7fb313a47700 1 -- 172.21.15.170:0/549818682 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7fb2ec028200 msgr2=0x7fb2ec02a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:36.559 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.557+0000 7fb313a47700 1 --2- 172.21.15.170:0/549818682 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7fb2ec028200 0x7fb2ec02a6c0 secure :-1 s=READY pgs=71 cs=0 l=1 rx=0x7fb2fc005500 tx=0x7fb2fc0016e0).stop 2020-03-19T21:18:36.559 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.557+0000 7fb313a47700 1 -- 172.21.15.170:0/549818682 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb30c125060 msgr2=0x7fb30c12f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:36.559 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.557+0000 7fb313a47700 1 --2- 172.21.15.170:0/549818682 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb30c125060 0x7fb30c12f2c0 secure :-1 s=READY pgs=289 cs=0 l=1 rx=0x7fb308002f30 tx=0x7fb308007a40).stop 2020-03-19T21:18:36.560 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.557+0000 7fb313a47700 1 -- 172.21.15.170:0/549818682 shutdown_connections 2020-03-19T21:18:36.560 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.557+0000 7fb313a47700 1 --2- 172.21.15.170:0/549818682 >> [v2:172.21.15.170:6816/20682,v1:172.21.15.170:6817/20682] conn(0x7fb30c056040 0x7fb30c056420 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:36.560 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.557+0000 7fb313a47700 1 --2- 172.21.15.170:0/549818682 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7fb2ec028200 0x7fb2ec02a6c0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:36.560 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.557+0000 7fb313a47700 1 --2- 172.21.15.170:0/549818682 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb30c125060 0x7fb30c12f2c0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:36.560 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.557+0000 7fb313a47700 1 -- 172.21.15.170:0/549818682 >> 172.21.15.170:0/549818682 conn(0x7fb30c091390 msgr2=0x7fb30c08dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:36.560 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.557+0000 7fb313a47700 1 -- 172.21.15.170:0/549818682 shutdown_connections 2020-03-19T21:18:36.560 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.557+0000 7fb313a47700 1 -- 172.21.15.170:0/549818682 wait complete. 2020-03-19T21:18:36.561 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2020-03-19T21:18:36.561 INFO:teuthology.orchestra.run.smithi170.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2020-03-19T21:18:36.561 INFO:teuthology.orchestra.run.smithi170.stdout:} 2020-03-19T21:18:36.568 INFO:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd deep-scrub 2 2020-03-19T21:18:36.675 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.673+0000 7feccc8c8700 1 Processor -- start 2020-03-19T21:18:36.675 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.673+0000 7feccc8c8700 1 -- start start 2020-03-19T21:18:36.680 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.674+0000 7feccc8c8700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fecc8091eb0 0x7fecc811f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T21:18:36.680 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.674+0000 7feccc8c8700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecc80911c0 con 0x7fecc811fab0 2020-03-19T21:18:36.681 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.674+0000 7feccc8c8700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecc8067130 con 0x7fecc8091eb0 2020-03-19T21:18:36.681 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.674+0000 7fecc659c700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fecc8091eb0 0x7fecc811f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:36.681 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.674+0000 7fecc5d9b700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fecc811fab0 0x7fecc8060110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:54128/0 (socket says 172.21.15.170:54128) 2020-03-19T21:18:36.681 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.674+0000 7fecc5d9b700 1 -- 172.21.15.170:0/3347022214 learned_addr learned my addr 172.21.15.170:0/3347022214 (peer_addr_for_me v1:172.21.15.170:0/0) 2020-03-19T21:18:36.681 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.674+0000 7fecc659c700 1 -- 172.21.15.170:0/3347022214 >> v1:172.21.15.170:6789/0 conn(0x7fecc811fab0 legacy=0x7fecc8060110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:36.681 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.674+0000 7fecc659c700 1 -- 172.21.15.170:0/3347022214 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecc811f340 con 0x7fecc8091eb0 2020-03-19T21:18:36.682 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.674+0000 7fecc559a700 1 -- 172.21.15.170:0/3347022214 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4202540331 0 0) 0x7fecc80911c0 con 0x7fecc811fab0 2020-03-19T21:18:36.682 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.674+0000 7fecc659c700 1 --2- 172.21.15.170:0/3347022214 >> v2:172.21.15.170:3300/0 conn(0x7fecc8091eb0 0x7fecc811f650 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7fecb8002f30 tx=0x7fecb8007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:36.682 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.674+0000 7fecc659c700 1 --2- 172.21.15.170:0/3347022214 >> v2:172.21.15.170:3300/0 conn(0x7fecc8091eb0 0x7fecc811f650 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7fecb8002f30 tx=0x7fecb8007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:36.682 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.675+0000 7fecc659c700 1 --2- 172.21.15.170:0/3347022214 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fecc8091eb0 0x7fecc811f650 secure :-1 s=READY pgs=291 cs=0 l=1 rx=0x7fecb8002f30 tx=0x7fecb8007a40).ready entity=mon.0 client_cookie=8816ff689b5aa4d8 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:36.682 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.675+0000 7fecc659c700 1 --2- 172.21.15.170:0/3347022214 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fecc8091eb0 0x7fecc811f650 secure :-1 s=THROTTLE_DONE pgs=291 cs=0 l=1 rx=0x7fecb8002f30 tx=0x7fecb8007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:36.682 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.675+0000 7fecc559a700 1 -- 172.21.15.170:0/3347022214 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fecb8008e80 con 0x7fecc8091eb0 2020-03-19T21:18:36.683 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.675+0000 7fecc659c700 1 --2- 172.21.15.170:0/3347022214 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fecc8091eb0 0x7fecc811f650 secure :-1 s=THROTTLE_DONE pgs=291 cs=0 l=1 rx=0x7fecb8002f30 tx=0x7fecb8007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:36.683 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.675+0000 7fecc559a700 1 -- 172.21.15.170:0/3347022214 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecb80113f0 con 0x7fecc8091eb0 2020-03-19T21:18:36.683 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.675+0000 7fecc659c700 1 --2- 172.21.15.170:0/3347022214 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fecc8091eb0 0x7fecc811f650 secure :-1 s=THROTTLE_DONE pgs=291 cs=0 l=1 rx=0x7fecb8002f30 tx=0x7fecb8007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:36.683 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.675+0000 7feccc8c8700 1 -- 172.21.15.170:0/3347022214 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fecc8091eb0 msgr2=0x7fecc811f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:36.683 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.675+0000 7fecc559a700 1 -- 172.21.15.170:0/3347022214 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fecb8011690 con 0x7fecc8091eb0 2020-03-19T21:18:36.683 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.675+0000 7feccc8c8700 1 --2- 172.21.15.170:0/3347022214 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fecc8091eb0 0x7fecc811f650 secure :-1 s=READY pgs=291 cs=0 l=1 rx=0x7fecb8002f30 tx=0x7fecb8007a40).stop 2020-03-19T21:18:36.683 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.675+0000 7feccc8c8700 1 -- 172.21.15.170:0/3347022214 shutdown_connections 2020-03-19T21:18:36.684 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.675+0000 7feccc8c8700 1 --2- 172.21.15.170:0/3347022214 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fecc8091eb0 0x7fecc811f650 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:36.684 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.675+0000 7feccc8c8700 1 -- 172.21.15.170:0/3347022214 >> 172.21.15.170:0/3347022214 conn(0x7fecc8091390 msgr2=0x7fecc80917a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:36.684 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.675+0000 7feccc8c8700 1 -- 172.21.15.170:0/3347022214 shutdown_connections 2020-03-19T21:18:36.684 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.675+0000 7feccc8c8700 1 -- 172.21.15.170:0/3347022214 wait complete. 2020-03-19T21:18:36.684 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.676+0000 7feccc8c8700 1 Processor -- start 2020-03-19T21:18:36.684 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.676+0000 7feccc8c8700 1 -- start start 2020-03-19T21:18:36.684 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.676+0000 7feccc8c8700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fecc811fab0 0x7fecc811c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:36.685 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.676+0000 7feccc8c8700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecc8120350 con 0x7fecc8091eb0 2020-03-19T21:18:36.685 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.676+0000 7feccc8c8700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecc8066540 con 0x7fecc811fab0 2020-03-19T21:18:36.685 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.676+0000 7fecc659c700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fecc8091eb0 0x7fecc811be60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:54132/0 (socket says 172.21.15.170:54132) 2020-03-19T21:18:36.685 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.676+0000 7fecc659c700 1 -- 172.21.15.170:0/770003620 learned_addr learned my addr 172.21.15.170:0/770003620 (peer_addr_for_me v1:172.21.15.170:0/0) 2020-03-19T21:18:36.685 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.676+0000 7fecc5d9b700 1 --2- 172.21.15.170:0/770003620 >> v2:172.21.15.170:3300/0 conn(0x7fecc811fab0 0x7fecc811c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:36.685 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.677+0000 7fecb77fe700 1 -- 172.21.15.170:0/770003620 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 834963351 0 0) 0x7fecc811f340 con 0x7fecc8091eb0 2020-03-19T21:18:36.685 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.677+0000 7fecb77fe700 1 -- 172.21.15.170:0/770003620 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feca4002fe0 con 0x7fecc8091eb0 2020-03-19T21:18:36.686 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.677+0000 7fecc5d9b700 1 -- 172.21.15.170:0/770003620 >> v1:172.21.15.170:6789/0 conn(0x7fecc8091eb0 legacy=0x7fecc811be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:36.686 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.677+0000 7fecc5d9b700 1 -- 172.21.15.170:0/770003620 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecc810d5d0 con 0x7fecc811fab0 2020-03-19T21:18:36.686 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.677+0000 7fecc5d9b700 1 --2- 172.21.15.170:0/770003620 >> v2:172.21.15.170:3300/0 conn(0x7fecc811fab0 0x7fecc811c570 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7fecc810b090 tx=0x7fecbc010860).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:36.686 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.677+0000 7fecc5d9b700 1 --2- 172.21.15.170:0/770003620 >> v2:172.21.15.170:3300/0 conn(0x7fecc811fab0 0x7fecc811c570 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7fecc810b090 tx=0x7fecbc010860).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:36.686 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.677+0000 7fecc5d9b700 1 --2- 172.21.15.170:0/770003620 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fecc811fab0 0x7fecc811c570 secure :-1 s=READY pgs=293 cs=0 l=1 rx=0x7fecc810b090 tx=0x7fecbc010860).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:36.687 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.677+0000 7fecc5d9b700 1 --2- 172.21.15.170:0/770003620 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fecc811fab0 0x7fecc811c570 secure :-1 s=THROTTLE_DONE pgs=293 cs=0 l=1 rx=0x7fecc810b090 tx=0x7fecbc010860).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:36.687 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.677+0000 7fecb77fe700 1 -- 172.21.15.170:0/770003620 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fecbc005bc0 con 0x7fecc811fab0 2020-03-19T21:18:36.687 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.677+0000 7fecc5d9b700 1 --2- 172.21.15.170:0/770003620 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fecc811fab0 0x7fecc811c570 secure :-1 s=THROTTLE_DONE pgs=293 cs=0 l=1 rx=0x7fecc810b090 tx=0x7fecbc010860).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:36.687 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.677+0000 7fecc5d9b700 1 --2- 172.21.15.170:0/770003620 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fecc811fab0 0x7fecc811c570 secure :-1 s=THROTTLE_DONE pgs=293 cs=0 l=1 rx=0x7fecc810b090 tx=0x7fecbc010860).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:36.687 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.677+0000 7fecb77fe700 1 -- 172.21.15.170:0/770003620 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecbc005d20 con 0x7fecc811fab0 2020-03-19T21:18:36.687 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.677+0000 7feccc8c8700 1 -- 172.21.15.170:0/770003620 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecc810dbe0 con 0x7fecc811fab0 2020-03-19T21:18:36.687 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.677+0000 7fecb77fe700 1 -- 172.21.15.170:0/770003620 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fecbc005e90 con 0x7fecc811fab0 2020-03-19T21:18:36.688 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.677+0000 7feccc8c8700 1 -- 172.21.15.170:0/770003620 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecc811c9d0 con 0x7fecc811fab0 2020-03-19T21:18:36.688 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.677+0000 7fecc5d9b700 1 --2- 172.21.15.170:0/770003620 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fecc811fab0 0x7fecc811c570 secure :-1 s=THROTTLE_DONE pgs=293 cs=0 l=1 rx=0x7fecc810b090 tx=0x7fecbc010860).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:36.688 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.677+0000 7fecb77fe700 1 -- 172.21.15.170:0/770003620 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 4) v1 ==== 29603+0+0 (secure 0 0 0) 0x7fecbc006370 con 0x7fecc811fab0 2020-03-19T21:18:36.688 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.677+0000 7fecc5d9b700 1 --2- 172.21.15.170:0/770003620 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fecc811fab0 0x7fecc811c570 secure :-1 s=THROTTLE_DONE pgs=293 cs=0 l=1 rx=0x7fecc810b090 tx=0x7fecbc010860).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:36.688 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.678+0000 7fecb77fe700 1 --2- 172.21.15.170:0/770003620 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7feca4028720 0x7feca402abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:36.688 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.678+0000 7fecb77fe700 1 -- 172.21.15.170:0/770003620 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 3786+0+0 (secure 0 0 0) 0x7fecbc034070 con 0x7fecc811fab0 2020-03-19T21:18:36.688 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.678+0000 7fecc659c700 1 --2- 172.21.15.170:0/770003620 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7feca4028720 0x7feca402abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:36.689 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.678+0000 7fecc659c700 1 --2- 172.21.15.170:0/770003620 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7feca4028720 0x7feca402abe0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7fecb8000bb0 tx=0x7fecb8007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:36.689 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.678+0000 7fecc659c700 1 --2- 172.21.15.170:0/770003620 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7feca4028720 0x7feca402abe0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7fecb8000bb0 tx=0x7fecb8007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:36.689 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.678+0000 7fecc659c700 1 --2- 172.21.15.170:0/770003620 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7feca4028720 0x7feca402abe0 secure :-1 s=READY pgs=72 cs=0 l=1 rx=0x7fecb8000bb0 tx=0x7fecb8007a40).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:36.689 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.678+0000 7feccc8c8700 1 -- 172.21.15.170:0/770003620 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecc810b6d0 con 0x7fecc811fab0 2020-03-19T21:18:36.690 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.681+0000 7fecc5d9b700 1 --2- 172.21.15.170:0/770003620 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fecc811fab0 0x7fecc811c570 secure :-1 s=THROTTLE_DONE pgs=293 cs=0 l=1 rx=0x7fecc810b090 tx=0x7fecbc010860).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:36.690 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.681+0000 7fecb77fe700 1 -- 172.21.15.170:0/770003620 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+126772 (secure 0 0 0) 0x7fecbc0090b0 con 0x7fecc811fab0 2020-03-19T21:18:36.821 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.820+0000 7feccc8c8700 1 -- 172.21.15.170:0/770003620 --> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7fecc810d410 con 0x7feca4028720 2020-03-19T21:18:36.822 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.820+0000 7fecc659c700 1 --2- 172.21.15.170:0/770003620 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7feca4028720 0x7feca402abe0 secure :-1 s=THROTTLE_DONE pgs=72 cs=0 l=1 rx=0x7fecb8000bb0 tx=0x7fecb8007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:36.822 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.820+0000 7fecb77fe700 1 -- 172.21.15.170:0/770003620 <== mgr.4102 v2:172.21.15.170:6824/20154 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fecc810d410 con 0x7feca4028720 2020-03-19T21:18:36.823 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.821+0000 7feccc8c8700 1 -- 172.21.15.170:0/770003620 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7feca4028720 msgr2=0x7feca402abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:36.823 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.821+0000 7feccc8c8700 1 --2- 172.21.15.170:0/770003620 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7feca4028720 0x7feca402abe0 secure :-1 s=READY pgs=72 cs=0 l=1 rx=0x7fecb8000bb0 tx=0x7fecb8007a40).stop 2020-03-19T21:18:36.823 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.822+0000 7feccc8c8700 1 -- 172.21.15.170:0/770003620 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fecc811fab0 msgr2=0x7fecc811c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:36.824 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.822+0000 7feccc8c8700 1 --2- 172.21.15.170:0/770003620 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fecc811fab0 0x7fecc811c570 secure :-1 s=READY pgs=293 cs=0 l=1 rx=0x7fecc810b090 tx=0x7fecbc010860).stop 2020-03-19T21:18:36.824 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.822+0000 7feccc8c8700 1 -- 172.21.15.170:0/770003620 shutdown_connections 2020-03-19T21:18:36.824 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.822+0000 7feccc8c8700 1 --2- 172.21.15.170:0/770003620 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7feca4028720 0x7feca402abe0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:36.824 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.822+0000 7feccc8c8700 1 --2- 172.21.15.170:0/770003620 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fecc811fab0 0x7fecc811c570 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:36.824 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.822+0000 7feccc8c8700 1 -- 172.21.15.170:0/770003620 >> 172.21.15.170:0/770003620 conn(0x7fecc8091390 msgr2=0x7fecc8060110 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:36.824 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.822+0000 7feccc8c8700 1 -- 172.21.15.170:0/770003620 shutdown_connections 2020-03-19T21:18:36.825 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.822+0000 7feccc8c8700 1 -- 172.21.15.170:0/770003620 wait complete. 2020-03-19T21:18:36.825 INFO:teuthology.orchestra.run.smithi170.stderr:instructed osd(s) 2 to deep-scrub 2020-03-19T21:18:36.833 INFO:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early pg dump --format=json 2020-03-19T21:18:36.944 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.937+0000 7eff7f932700 1 Processor -- start 2020-03-19T21:18:36.944 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.937+0000 7eff7f932700 1 -- start start 2020-03-19T21:18:36.944 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.937+0000 7eff7f932700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7eff78065d30 0x7eff78066110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T21:18:36.945 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.937+0000 7eff7f932700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff78090af0 con 0x7eff78066570 2020-03-19T21:18:36.945 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.937+0000 7eff7f932700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff7808eec0 con 0x7eff78065d30 2020-03-19T21:18:36.945 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.938+0000 7eff7d6ce700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7eff78065d30 0x7eff78066110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:36.945 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.938+0000 7eff7cecd700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7eff78066570 0x7eff7808d000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:54140/0 (socket says 172.21.15.170:54140) 2020-03-19T21:18:36.945 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.938+0000 7eff7cecd700 1 -- 172.21.15.170:0/2262029886 learned_addr learned my addr 172.21.15.170:0/2262029886 (peer_addr_for_me v1:172.21.15.170:0/0) 2020-03-19T21:18:36.946 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.938+0000 7eff7d6ce700 1 -- 172.21.15.170:0/2262029886 >> v1:172.21.15.170:6789/0 conn(0x7eff78066570 legacy=0x7eff7808d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:36.946 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.938+0000 7eff7d6ce700 1 -- 172.21.15.170:0/2262029886 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff78060ae0 con 0x7eff78065d30 2020-03-19T21:18:36.946 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.938+0000 7eff6bfff700 1 -- 172.21.15.170:0/2262029886 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 161110555 0 0) 0x7eff78090af0 con 0x7eff78066570 2020-03-19T21:18:36.946 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.938+0000 7eff7d6ce700 1 --2- 172.21.15.170:0/2262029886 >> v2:172.21.15.170:3300/0 conn(0x7eff78065d30 0x7eff78066110 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7eff700032b0 tx=0x7eff70007dc0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:36.946 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.938+0000 7eff7d6ce700 1 --2- 172.21.15.170:0/2262029886 >> v2:172.21.15.170:3300/0 conn(0x7eff78065d30 0x7eff78066110 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7eff700032b0 tx=0x7eff70007dc0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:36.946 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.938+0000 7eff7d6ce700 1 --2- 172.21.15.170:0/2262029886 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff78065d30 0x7eff78066110 secure :-1 s=READY pgs=295 cs=0 l=1 rx=0x7eff700032b0 tx=0x7eff70007dc0).ready entity=mon.0 client_cookie=72264b6217a6ada9 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:36.946 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.938+0000 7eff7d6ce700 1 --2- 172.21.15.170:0/2262029886 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff78065d30 0x7eff78066110 secure :-1 s=THROTTLE_DONE pgs=295 cs=0 l=1 rx=0x7eff700032b0 tx=0x7eff70007dc0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:36.947 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.938+0000 7eff6bfff700 1 -- 172.21.15.170:0/2262029886 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7eff70011cd0 con 0x7eff78065d30 2020-03-19T21:18:36.947 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.938+0000 7eff7d6ce700 1 --2- 172.21.15.170:0/2262029886 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff78065d30 0x7eff78066110 secure :-1 s=THROTTLE_DONE pgs=295 cs=0 l=1 rx=0x7eff700032b0 tx=0x7eff70007dc0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:36.947 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.938+0000 7eff6bfff700 1 -- 172.21.15.170:0/2262029886 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff70011e30 con 0x7eff78065d30 2020-03-19T21:18:36.947 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.938+0000 7eff7d6ce700 1 --2- 172.21.15.170:0/2262029886 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff78065d30 0x7eff78066110 secure :-1 s=THROTTLE_DONE pgs=295 cs=0 l=1 rx=0x7eff700032b0 tx=0x7eff70007dc0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:36.948 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.938+0000 7eff6bfff700 1 -- 172.21.15.170:0/2262029886 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7eff7000a290 con 0x7eff78065d30 2020-03-19T21:18:36.948 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.938+0000 7eff7f932700 1 -- 172.21.15.170:0/2262029886 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff78065d30 msgr2=0x7eff78066110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:36.948 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.938+0000 7eff7f932700 1 --2- 172.21.15.170:0/2262029886 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff78065d30 0x7eff78066110 secure :-1 s=READY pgs=295 cs=0 l=1 rx=0x7eff700032b0 tx=0x7eff70007dc0).stop 2020-03-19T21:18:36.948 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.939+0000 7eff7f932700 1 -- 172.21.15.170:0/2262029886 shutdown_connections 2020-03-19T21:18:36.948 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.939+0000 7eff7f932700 1 --2- 172.21.15.170:0/2262029886 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff78065d30 0x7eff78066110 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:36.948 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.939+0000 7eff7f932700 1 -- 172.21.15.170:0/2262029886 >> 172.21.15.170:0/2262029886 conn(0x7eff78091540 msgr2=0x7eff78091950 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:36.949 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.939+0000 7eff7f932700 1 -- 172.21.15.170:0/2262029886 shutdown_connections 2020-03-19T21:18:36.949 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.939+0000 7eff7f932700 1 -- 172.21.15.170:0/2262029886 wait complete. 2020-03-19T21:18:36.949 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.939+0000 7eff7f932700 1 Processor -- start 2020-03-19T21:18:36.949 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.939+0000 7eff7f932700 1 -- start start 2020-03-19T21:18:36.949 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.940+0000 7eff7f932700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7eff78065d30 0x7eff780848d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:36.949 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.940+0000 7eff7f932700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff78088250 con 0x7eff78066570 2020-03-19T21:18:36.950 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.940+0000 7eff7f932700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff78086e30 con 0x7eff78065d30 2020-03-19T21:18:36.950 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.940+0000 7eff7d6ce700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7eff78065d30 0x7eff780848d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:36.950 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.940+0000 7eff7cecd700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7eff78066570 0x7eff78084d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:54144/0 (socket says 172.21.15.170:54144) 2020-03-19T21:18:36.950 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.940+0000 7eff7cecd700 1 -- 172.21.15.170:0/531179592 learned_addr learned my addr 172.21.15.170:0/531179592 (peer_addr_for_me v1:172.21.15.170:0/0) 2020-03-19T21:18:36.950 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.940+0000 7eff7d6ce700 1 -- 172.21.15.170:0/531179592 >> v1:172.21.15.170:6789/0 conn(0x7eff78066570 legacy=0x7eff78084d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:36.950 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.940+0000 7eff6a7fc700 1 -- 172.21.15.170:0/531179592 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3979811402 0 0) 0x7eff78088250 con 0x7eff78066570 2020-03-19T21:18:36.950 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.940+0000 7eff7d6ce700 1 -- 172.21.15.170:0/531179592 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff78087020 con 0x7eff78065d30 2020-03-19T21:18:36.951 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.940+0000 7eff7d6ce700 1 --2- 172.21.15.170:0/531179592 >> v2:172.21.15.170:3300/0 conn(0x7eff78065d30 0x7eff780848d0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7eff700032b0 tx=0x7eff70007dc0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:36.951 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.940+0000 7eff7d6ce700 1 --2- 172.21.15.170:0/531179592 >> v2:172.21.15.170:3300/0 conn(0x7eff78065d30 0x7eff780848d0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7eff700032b0 tx=0x7eff70007dc0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:36.951 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.940+0000 7eff7d6ce700 1 --2- 172.21.15.170:0/531179592 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff78065d30 0x7eff780848d0 secure :-1 s=READY pgs=297 cs=0 l=1 rx=0x7eff700032b0 tx=0x7eff70007dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:36.951 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.940+0000 7eff7d6ce700 1 --2- 172.21.15.170:0/531179592 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff78065d30 0x7eff780848d0 secure :-1 s=THROTTLE_DONE pgs=297 cs=0 l=1 rx=0x7eff700032b0 tx=0x7eff70007dc0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:36.951 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.940+0000 7eff6a7fc700 1 -- 172.21.15.170:0/531179592 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7eff70005a40 con 0x7eff78065d30 2020-03-19T21:18:36.952 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.940+0000 7eff7d6ce700 1 --2- 172.21.15.170:0/531179592 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff78065d30 0x7eff780848d0 secure :-1 s=THROTTLE_DONE pgs=297 cs=0 l=1 rx=0x7eff700032b0 tx=0x7eff70007dc0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:36.952 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.940+0000 7eff7d6ce700 1 --2- 172.21.15.170:0/531179592 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff78065d30 0x7eff780848d0 secure :-1 s=THROTTLE_DONE pgs=297 cs=0 l=1 rx=0x7eff700032b0 tx=0x7eff70007dc0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:36.952 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.940+0000 7eff6a7fc700 1 -- 172.21.15.170:0/531179592 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff70005ba0 con 0x7eff78065d30 2020-03-19T21:18:36.952 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.940+0000 7eff6a7fc700 1 -- 172.21.15.170:0/531179592 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7eff70005e40 con 0x7eff78065d30 2020-03-19T21:18:36.952 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.940+0000 7eff7f932700 1 -- 172.21.15.170:0/531179592 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff78086210 con 0x7eff78065d30 2020-03-19T21:18:36.952 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.941+0000 7eff7f932700 1 -- 172.21.15.170:0/531179592 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff780868b0 con 0x7eff78065d30 2020-03-19T21:18:36.952 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.941+0000 7eff7d6ce700 1 --2- 172.21.15.170:0/531179592 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff78065d30 0x7eff780848d0 secure :-1 s=THROTTLE_DONE pgs=297 cs=0 l=1 rx=0x7eff700032b0 tx=0x7eff70007dc0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:36.953 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.941+0000 7eff6a7fc700 1 -- 172.21.15.170:0/531179592 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 4) v1 ==== 29603+0+0 (secure 0 0 0) 0x7eff7000a6e0 con 0x7eff78065d30 2020-03-19T21:18:36.953 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.941+0000 7eff7d6ce700 1 --2- 172.21.15.170:0/531179592 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff78065d30 0x7eff780848d0 secure :-1 s=THROTTLE_DONE pgs=297 cs=0 l=1 rx=0x7eff700032b0 tx=0x7eff70007dc0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:36.953 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.941+0000 7eff6a7fc700 1 --2- 172.21.15.170:0/531179592 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7eff58028200 0x7eff5802a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:36.953 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.941+0000 7eff6a7fc700 1 -- 172.21.15.170:0/531179592 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 3786+0+0 (secure 0 0 0) 0x7eff70034f70 con 0x7eff78065d30 2020-03-19T21:18:36.953 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.941+0000 7eff7cecd700 1 --2- 172.21.15.170:0/531179592 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7eff58028200 0x7eff5802a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:36.953 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.941+0000 7eff7cecd700 1 --2- 172.21.15.170:0/531179592 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7eff58028200 0x7eff5802a6c0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7eff6c005500 tx=0x7eff6c001780).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:36.954 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.941+0000 7eff7cecd700 1 --2- 172.21.15.170:0/531179592 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7eff58028200 0x7eff5802a6c0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7eff6c005500 tx=0x7eff6c001780).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:36.954 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.942+0000 7eff7cecd700 1 --2- 172.21.15.170:0/531179592 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7eff58028200 0x7eff5802a6c0 secure :-1 s=READY pgs=73 cs=0 l=1 rx=0x7eff6c005500 tx=0x7eff6c001780).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:36.954 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.942+0000 7eff7f932700 1 -- 172.21.15.170:0/531179592 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff78084260 con 0x7eff78065d30 2020-03-19T21:18:36.954 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.945+0000 7eff7d6ce700 1 --2- 172.21.15.170:0/531179592 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff78065d30 0x7eff780848d0 secure :-1 s=THROTTLE_DONE pgs=297 cs=0 l=1 rx=0x7eff700032b0 tx=0x7eff70007dc0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:36.954 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:36.945+0000 7eff6a7fc700 1 -- 172.21.15.170:0/531179592 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+126772 (secure 0 0 0) 0x7eff70006ad0 con 0x7eff78065d30 2020-03-19T21:18:37.078 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:37.077+0000 7eff7f932700 1 -- 172.21.15.170:0/531179592 --> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7eff78056db0 con 0x7eff58028200 2020-03-19T21:18:37.082 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:37.079+0000 7eff7cecd700 1 --2- 172.21.15.170:0/531179592 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7eff58028200 0x7eff5802a6c0 secure :-1 s=THROTTLE_DONE pgs=73 cs=0 l=1 rx=0x7eff6c005500 tx=0x7eff6c001780).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:37.082 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:37.079+0000 7eff6a7fc700 1 -- 172.21.15.170:0/531179592 <== mgr.4102 v2:172.21.15.170:6824/20154 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+74054 (secure 0 0 0) 0x7eff78056db0 con 0x7eff58028200 2020-03-19T21:18:37.082 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:37.081+0000 7eff7f932700 1 -- 172.21.15.170:0/531179592 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7eff58028200 msgr2=0x7eff5802a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:37.083 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:37.081+0000 7eff7f932700 1 --2- 172.21.15.170:0/531179592 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7eff58028200 0x7eff5802a6c0 secure :-1 s=READY pgs=73 cs=0 l=1 rx=0x7eff6c005500 tx=0x7eff6c001780).stop 2020-03-19T21:18:37.083 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:37.081+0000 7eff7f932700 1 -- 172.21.15.170:0/531179592 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff78065d30 msgr2=0x7eff780848d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:37.083 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:37.081+0000 7eff7f932700 1 --2- 172.21.15.170:0/531179592 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff78065d30 0x7eff780848d0 secure :-1 s=READY pgs=297 cs=0 l=1 rx=0x7eff700032b0 tx=0x7eff70007dc0).stop 2020-03-19T21:18:37.083 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:37.081+0000 7eff7f932700 1 -- 172.21.15.170:0/531179592 shutdown_connections 2020-03-19T21:18:37.083 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:37.081+0000 7eff7f932700 1 --2- 172.21.15.170:0/531179592 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7eff58028200 0x7eff5802a6c0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:37.083 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:37.081+0000 7eff7f932700 1 --2- 172.21.15.170:0/531179592 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff78065d30 0x7eff780848d0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:37.084 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:37.081+0000 7eff7f932700 1 -- 172.21.15.170:0/531179592 >> 172.21.15.170:0/531179592 conn(0x7eff78091540 msgr2=0x7eff78060c70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:37.084 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:37.081+0000 7eff7f932700 1 -- 172.21.15.170:0/531179592 shutdown_connections 2020-03-19T21:18:37.084 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:37.081+0000 7eff7f932700 1 -- 172.21.15.170:0/531179592 wait complete. 2020-03-19T21:18:37.084 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:18:37.086 INFO:teuthology.orchestra.run.smithi170.stderr:dumped all 2020-03-19T21:18:37.092 INFO:teuthology.orchestra.run.smithi170.stdout:{"pg_ready":true,"pg_map":{"version":38,"stamp":"2020-03-19T21:18:36.512802+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2718,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":314572800,"kb_used":3146416,"kb_used_data":664,"kb_used_omap":0,"kb_used_meta":3145728,"kb_avail":311426384,"statfs":{"total":322122547200,"available":318900617216,"internally_reserved":3221225472,"allocated":679936,"data_stored":238689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":3221225472},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":432,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001398"},"pg_stats":[{"pgid":"4.0","version":"0'0","reported_seq":"13","reported_epoch":"23","state":"active+clean","last_fresh":"2020-03-19T21:18:29.507255+0000","last_change":"2020-03-19T21:18:24.224546+0000","last_active":"2020-03-19T21:18:29.507255+0000","last_peered":"2020-03-19T21:18:29.507255+0000","last_clean":"2020-03-19T21:18:29.507255+0000","last_became_active":"2020-03-19T21:18:24.224136+0000","last_became_peered":"2020-03-19T21:18:24.224136+0000","last_unstale":"2020-03-19T21:18:29.507255+0000","last_undegraded":"2020-03-19T21:18:29.507255+0000","last_fullsized":"2020-03-19T21:18:29.507255+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:23.204329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:23.204329+0000","last_clean_scrub_stamp":"2020-03-19T21:18:23.204329+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":"16","reported_epoch":"23","state":"active+clean","last_fresh":"2020-03-19T21:18:30.007436+0000","last_change":"2020-03-19T21:18:13.525398+0000","last_active":"2020-03-19T21:18:30.007436+0000","last_peered":"2020-03-19T21:18:30.007436+0000","last_clean":"2020-03-19T21:18:30.007436+0000","last_became_active":"2020-03-19T21:18:13.525041+0000","last_became_peered":"2020-03-19T21:18:13.525041+0000","last_unstale":"2020-03-19T21:18:30.007436+0000","last_undegraded":"2020-03-19T21:18:30.007436+0000","last_fullsized":"2020-03-19T21:18:30.007436+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:13.185694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:13.185694+0000","last_clean_scrub_stamp":"2020-03-19T21:18:13.185694+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":"20","reported_epoch":"23","state":"active+clean","last_fresh":"2020-03-19T21:18:30.007878+0000","last_change":"2020-03-19T21:18:10.186965+0000","last_active":"2020-03-19T21:18:30.007878+0000","last_peered":"2020-03-19T21:18:30.007878+0000","last_clean":"2020-03-19T21:18:30.007878+0000","last_became_active":"2020-03-19T21:18:10.186584+0000","last_became_peered":"2020-03-19T21:18:10.186584+0000","last_unstale":"2020-03-19T21:18:30.007878+0000","last_undegraded":"2020-03-19T21:18:30.007878+0000","last_fullsized":"2020-03-19T21:18:30.007878+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:09.169195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:09.169195+0000","last_clean_scrub_stamp":"2020-03-19T21:18:09.169195+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"16'4","reported_seq":"22","reported_epoch":"23","state":"active+clean","last_fresh":"2020-03-19T21:18:29.245838+0000","last_change":"2020-03-19T21:18:13.197989+0000","last_active":"2020-03-19T21:18:29.245838+0000","last_peered":"2020-03-19T21:18:29.245838+0000","last_clean":"2020-03-19T21:18:29.245838+0000","last_became_active":"2020-03-19T21:18:13.197671+0000","last_became_peered":"2020-03-19T21:18:13.197671+0000","last_unstale":"2020-03-19T21:18:29.245838+0000","last_undegraded":"2020-03-19T21:18:29.245838+0000","last_fullsized":"2020-03-19T21:18:29.245838+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:12.182524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:12.182524+0000","last_clean_scrub_stamp":"2020-03-19T21:18:12.182524+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":"13","reported_epoch":"23","state":"active+clean","last_fresh":"2020-03-19T21:18:30.007827+0000","last_change":"2020-03-19T21:18:24.224285+0000","last_active":"2020-03-19T21:18:30.007827+0000","last_peered":"2020-03-19T21:18:30.007827+0000","last_clean":"2020-03-19T21:18:30.007827+0000","last_became_active":"2020-03-19T21:18:24.224025+0000","last_became_peered":"2020-03-19T21:18:24.224025+0000","last_unstale":"2020-03-19T21:18:30.007827+0000","last_undegraded":"2020-03-19T21:18:30.007827+0000","last_fullsized":"2020-03-19T21:18:30.007827+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:23.204329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:23.204329+0000","last_clean_scrub_stamp":"2020-03-19T21:18:23.204329+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":"16","reported_epoch":"23","state":"active+clean","last_fresh":"2020-03-19T21:18:29.507366+0000","last_change":"2020-03-19T21:18:13.526027+0000","last_active":"2020-03-19T21:18:29.507366+0000","last_peered":"2020-03-19T21:18:29.507366+0000","last_clean":"2020-03-19T21:18:29.507366+0000","last_became_active":"2020-03-19T21:18:13.524563+0000","last_became_peered":"2020-03-19T21:18:13.524563+0000","last_unstale":"2020-03-19T21:18:29.507366+0000","last_undegraded":"2020-03-19T21:18:29.507366+0000","last_fullsized":"2020-03-19T21:18:29.507366+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:13.185694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:13.185694+0000","last_clean_scrub_stamp":"2020-03-19T21:18:13.185694+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":"20","reported_epoch":"23","state":"active+clean","last_fresh":"2020-03-19T21:18:29.245592+0000","last_change":"2020-03-19T21:18:10.184204+0000","last_active":"2020-03-19T21:18:29.245592+0000","last_peered":"2020-03-19T21:18:29.245592+0000","last_clean":"2020-03-19T21:18:29.245592+0000","last_became_active":"2020-03-19T21:18:10.183810+0000","last_became_peered":"2020-03-19T21:18:10.183810+0000","last_unstale":"2020-03-19T21:18:29.245592+0000","last_undegraded":"2020-03-19T21:18:29.245592+0000","last_fullsized":"2020-03-19T21:18:29.245592+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:09.169195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:09.169195+0000","last_clean_scrub_stamp":"2020-03-19T21:18:09.169195+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"16'7","reported_seq":"25","reported_epoch":"23","state":"active+clean","last_fresh":"2020-03-19T21:18:29.245983+0000","last_change":"2020-03-19T21:18:13.196212+0000","last_active":"2020-03-19T21:18:29.245983+0000","last_peered":"2020-03-19T21:18:29.245983+0000","last_clean":"2020-03-19T21:18:29.245983+0000","last_became_active":"2020-03-19T21:18:13.195896+0000","last_became_peered":"2020-03-19T21:18:13.195896+0000","last_unstale":"2020-03-19T21:18:29.245983+0000","last_undegraded":"2020-03-19T21:18:29.245983+0000","last_fullsized":"2020-03-19T21:18:29.245983+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:12.182524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:12.182524+0000","last_clean_scrub_stamp":"2020-03-19T21:18:12.182524+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":566,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":"16","reported_epoch":"23","state":"active+clean","last_fresh":"2020-03-19T21:18:29.507255+0000","last_change":"2020-03-19T21:18:13.524166+0000","last_active":"2020-03-19T21:18:29.507255+0000","last_peered":"2020-03-19T21:18:29.507255+0000","last_clean":"2020-03-19T21:18:29.507255+0000","last_became_active":"2020-03-19T21:18:13.523806+0000","last_became_peered":"2020-03-19T21:18:13.523806+0000","last_unstale":"2020-03-19T21:18:29.507255+0000","last_undegraded":"2020-03-19T21:18:29.507255+0000","last_fullsized":"2020-03-19T21:18:29.507255+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:13.185694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:13.185694+0000","last_clean_scrub_stamp":"2020-03-19T21:18:13.185694+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":"20","reported_epoch":"23","state":"active+clean","last_fresh":"2020-03-19T21:18:29.245663+0000","last_change":"2020-03-19T21:18:10.184210+0000","last_active":"2020-03-19T21:18:29.245663+0000","last_peered":"2020-03-19T21:18:29.245663+0000","last_clean":"2020-03-19T21:18:29.245663+0000","last_became_active":"2020-03-19T21:18:10.183788+0000","last_became_peered":"2020-03-19T21:18:10.183788+0000","last_unstale":"2020-03-19T21:18:29.245663+0000","last_undegraded":"2020-03-19T21:18:29.245663+0000","last_fullsized":"2020-03-19T21:18:29.245663+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:09.169195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:09.169195+0000","last_clean_scrub_stamp":"2020-03-19T21:18:09.169195+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"16'8","reported_seq":"25","reported_epoch":"23","state":"active+clean","last_fresh":"2020-03-19T21:18:30.007670+0000","last_change":"2020-03-19T21:18:13.197291+0000","last_active":"2020-03-19T21:18:30.007670+0000","last_peered":"2020-03-19T21:18:30.007670+0000","last_clean":"2020-03-19T21:18:30.007670+0000","last_became_active":"2020-03-19T21:18:13.196823+0000","last_became_peered":"2020-03-19T21:18:13.196823+0000","last_unstale":"2020-03-19T21:18:30.007670+0000","last_undegraded":"2020-03-19T21:18:30.007670+0000","last_fullsized":"2020-03-19T21:18:30.007670+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:12.182524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:12.182524+0000","last_clean_scrub_stamp":"2020-03-19T21:18:12.182524+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":"17","reported_epoch":"23","state":"active+clean","last_fresh":"2020-03-19T21:18:29.245702+0000","last_change":"2020-03-19T21:18:13.528239+0000","last_active":"2020-03-19T21:18:29.245702+0000","last_peered":"2020-03-19T21:18:29.245702+0000","last_clean":"2020-03-19T21:18:29.245702+0000","last_became_active":"2020-03-19T21:18:13.527916+0000","last_became_peered":"2020-03-19T21:18:13.527916+0000","last_unstale":"2020-03-19T21:18:29.245702+0000","last_undegraded":"2020-03-19T21:18:29.245702+0000","last_fullsized":"2020-03-19T21:18:29.245702+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:13.185694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:13.185694+0000","last_clean_scrub_stamp":"2020-03-19T21:18:13.185694+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":"19","reported_epoch":"23","state":"active+clean","last_fresh":"2020-03-19T21:18:29.507239+0000","last_change":"2020-03-19T21:18:10.187875+0000","last_active":"2020-03-19T21:18:29.507239+0000","last_peered":"2020-03-19T21:18:29.507239+0000","last_clean":"2020-03-19T21:18:29.507239+0000","last_became_active":"2020-03-19T21:18:10.187570+0000","last_became_peered":"2020-03-19T21:18:10.187570+0000","last_unstale":"2020-03-19T21:18:29.507239+0000","last_undegraded":"2020-03-19T21:18:29.507239+0000","last_fullsized":"2020-03-19T21:18:29.507239+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:09.169195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:09.169195+0000","last_clean_scrub_stamp":"2020-03-19T21:18:09.169195+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"17","reported_epoch":"23","state":"active+clean","last_fresh":"2020-03-19T21:18:30.008133+0000","last_change":"2020-03-19T21:18:13.197535+0000","last_active":"2020-03-19T21:18:30.008133+0000","last_peered":"2020-03-19T21:18:30.008133+0000","last_clean":"2020-03-19T21:18:30.008133+0000","last_became_active":"2020-03-19T21:18:13.197224+0000","last_became_peered":"2020-03-19T21:18:13.197224+0000","last_unstale":"2020-03-19T21:18:30.008133+0000","last_undegraded":"2020-03-19T21:18:30.008133+0000","last_fullsized":"2020-03-19T21:18:30.008133+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:12.182524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:12.182524+0000","last_clean_scrub_stamp":"2020-03-19T21:18:12.182524+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":"17","reported_epoch":"23","state":"active+clean","last_fresh":"2020-03-19T21:18:29.245873+0000","last_change":"2020-03-19T21:18:13.524218+0000","last_active":"2020-03-19T21:18:29.245873+0000","last_peered":"2020-03-19T21:18:29.245873+0000","last_clean":"2020-03-19T21:18:29.245873+0000","last_became_active":"2020-03-19T21:18:13.523902+0000","last_became_peered":"2020-03-19T21:18:13.523902+0000","last_unstale":"2020-03-19T21:18:29.245873+0000","last_undegraded":"2020-03-19T21:18:29.245873+0000","last_fullsized":"2020-03-19T21:18:29.245873+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:13.185694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:13.185694+0000","last_clean_scrub_stamp":"2020-03-19T21:18:13.185694+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":"20","reported_epoch":"23","state":"active+clean","last_fresh":"2020-03-19T21:18:29.245602+0000","last_change":"2020-03-19T21:18:10.184801+0000","last_active":"2020-03-19T21:18:29.245602+0000","last_peered":"2020-03-19T21:18:29.245602+0000","last_clean":"2020-03-19T21:18:29.245602+0000","last_became_active":"2020-03-19T21:18:10.184433+0000","last_became_peered":"2020-03-19T21:18:10.184433+0000","last_unstale":"2020-03-19T21:18:29.245602+0000","last_undegraded":"2020-03-19T21:18:29.245602+0000","last_fullsized":"2020-03-19T21:18:29.245602+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:09.169195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:09.169195+0000","last_clean_scrub_stamp":"2020-03-19T21:18:09.169195+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"16'5","reported_seq":"23","reported_epoch":"23","state":"active+clean","last_fresh":"2020-03-19T21:18:29.245548+0000","last_change":"2020-03-19T21:18:13.195202+0000","last_active":"2020-03-19T21:18:29.245548+0000","last_peered":"2020-03-19T21:18:29.245548+0000","last_clean":"2020-03-19T21:18:29.245548+0000","last_became_active":"2020-03-19T21:18:13.194849+0000","last_became_peered":"2020-03-19T21:18:13.194849+0000","last_unstale":"2020-03-19T21:18:29.245548+0000","last_undegraded":"2020-03-19T21:18:29.245548+0000","last_fullsized":"2020-03-19T21:18:29.245548+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:12.182524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:12.182524+0000","last_clean_scrub_stamp":"2020-03-19T21:18:12.182524+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":"16","reported_epoch":"23","state":"active+clean","last_fresh":"2020-03-19T21:18:29.507298+0000","last_change":"2020-03-19T21:18:13.523284+0000","last_active":"2020-03-19T21:18:29.507298+0000","last_peered":"2020-03-19T21:18:29.507298+0000","last_clean":"2020-03-19T21:18:29.507298+0000","last_became_active":"2020-03-19T21:18:13.522937+0000","last_became_peered":"2020-03-19T21:18:13.522937+0000","last_unstale":"2020-03-19T21:18:29.507298+0000","last_undegraded":"2020-03-19T21:18:29.507298+0000","last_fullsized":"2020-03-19T21:18:29.507298+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:13.185694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:13.185694+0000","last_clean_scrub_stamp":"2020-03-19T21:18:13.185694+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":"20","reported_epoch":"23","state":"active+clean","last_fresh":"2020-03-19T21:18:30.008123+0000","last_change":"2020-03-19T21:18:10.183717+0000","last_active":"2020-03-19T21:18:30.008123+0000","last_peered":"2020-03-19T21:18:30.008123+0000","last_clean":"2020-03-19T21:18:30.008123+0000","last_became_active":"2020-03-19T21:18:10.183404+0000","last_became_peered":"2020-03-19T21:18:10.183404+0000","last_unstale":"2020-03-19T21:18:30.008123+0000","last_undegraded":"2020-03-19T21:18:30.008123+0000","last_fullsized":"2020-03-19T21:18:30.008123+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:09.169195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:09.169195+0000","last_clean_scrub_stamp":"2020-03-19T21:18:09.169195+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"16'1","reported_seq":"18","reported_epoch":"23","state":"active+clean","last_fresh":"2020-03-19T21:18:29.507380+0000","last_change":"2020-03-19T21:18:13.198955+0000","last_active":"2020-03-19T21:18:29.507380+0000","last_peered":"2020-03-19T21:18:29.507380+0000","last_clean":"2020-03-19T21:18:29.507380+0000","last_became_active":"2020-03-19T21:18:13.198498+0000","last_became_peered":"2020-03-19T21:18:13.198498+0000","last_unstale":"2020-03-19T21:18:29.507380+0000","last_undegraded":"2020-03-19T21:18:29.507380+0000","last_fullsized":"2020-03-19T21:18:29.507380+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:12.182524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:12.182524+0000","last_clean_scrub_stamp":"2020-03-19T21:18:12.182524+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":"13","reported_epoch":"23","state":"active+clean","last_fresh":"2020-03-19T21:18:29.507226+0000","last_change":"2020-03-19T21:18:24.224701+0000","last_active":"2020-03-19T21:18:29.507226+0000","last_peered":"2020-03-19T21:18:29.507226+0000","last_clean":"2020-03-19T21:18:29.507226+0000","last_became_active":"2020-03-19T21:18:24.224328+0000","last_became_peered":"2020-03-19T21:18:24.224328+0000","last_unstale":"2020-03-19T21:18:29.507226+0000","last_undegraded":"2020-03-19T21:18:29.507226+0000","last_fullsized":"2020-03-19T21:18:29.507226+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:23.204329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:23.204329+0000","last_clean_scrub_stamp":"2020-03-19T21:18:23.204329+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":"17","reported_epoch":"23","state":"active+clean","last_fresh":"2020-03-19T21:18:29.245491+0000","last_change":"2020-03-19T21:18:13.524754+0000","last_active":"2020-03-19T21:18:29.245491+0000","last_peered":"2020-03-19T21:18:29.245491+0000","last_clean":"2020-03-19T21:18:29.245491+0000","last_became_active":"2020-03-19T21:18:13.524437+0000","last_became_peered":"2020-03-19T21:18:13.524437+0000","last_unstale":"2020-03-19T21:18:29.245491+0000","last_undegraded":"2020-03-19T21:18:29.245491+0000","last_fullsized":"2020-03-19T21:18:29.245491+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:13.185694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:13.185694+0000","last_clean_scrub_stamp":"2020-03-19T21:18:13.185694+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":"20","reported_epoch":"23","state":"active+clean","last_fresh":"2020-03-19T21:18:29.245786+0000","last_change":"2020-03-19T21:18:10.184911+0000","last_active":"2020-03-19T21:18:29.245786+0000","last_peered":"2020-03-19T21:18:29.245786+0000","last_clean":"2020-03-19T21:18:29.245786+0000","last_became_active":"2020-03-19T21:18:10.184549+0000","last_became_peered":"2020-03-19T21:18:10.184549+0000","last_unstale":"2020-03-19T21:18:29.245786+0000","last_undegraded":"2020-03-19T21:18:29.245786+0000","last_fullsized":"2020-03-19T21:18:29.245786+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:09.169195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:09.169195+0000","last_clean_scrub_stamp":"2020-03-19T21:18:09.169195+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"16'6","reported_seq":"24","reported_epoch":"23","state":"active+clean","last_fresh":"2020-03-19T21:18:29.245726+0000","last_change":"2020-03-19T21:18:13.196469+0000","last_active":"2020-03-19T21:18:29.245726+0000","last_peered":"2020-03-19T21:18:29.245726+0000","last_clean":"2020-03-19T21:18:29.245726+0000","last_became_active":"2020-03-19T21:18:13.195947+0000","last_became_peered":"2020-03-19T21:18:13.195947+0000","last_unstale":"2020-03-19T21:18:29.245726+0000","last_undegraded":"2020-03-19T21:18:29.245726+0000","last_fullsized":"2020-03-19T21:18:29.245726+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:12.182524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:12.182524+0000","last_clean_scrub_stamp":"2020-03-19T21:18:12.182524+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":160,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":"14","reported_epoch":"23","state":"active+clean","last_fresh":"2020-03-19T21:18:29.245638+0000","last_change":"2020-03-19T21:18:24.223995+0000","last_active":"2020-03-19T21:18:29.245638+0000","last_peered":"2020-03-19T21:18:29.245638+0000","last_clean":"2020-03-19T21:18:29.245638+0000","last_became_active":"2020-03-19T21:18:24.223594+0000","last_became_peered":"2020-03-19T21:18:24.223594+0000","last_unstale":"2020-03-19T21:18:29.245638+0000","last_undegraded":"2020-03-19T21:18:29.245638+0000","last_fullsized":"2020-03-19T21:18:29.245638+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:23.204329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:23.204329+0000","last_clean_scrub_stamp":"2020-03-19T21:18:23.204329+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":"16","reported_epoch":"23","state":"active+clean","last_fresh":"2020-03-19T21:18:30.007327+0000","last_change":"2020-03-19T21:18:13.525236+0000","last_active":"2020-03-19T21:18:30.007327+0000","last_peered":"2020-03-19T21:18:30.007327+0000","last_clean":"2020-03-19T21:18:30.007327+0000","last_became_active":"2020-03-19T21:18:13.524938+0000","last_became_peered":"2020-03-19T21:18:13.524938+0000","last_unstale":"2020-03-19T21:18:30.007327+0000","last_undegraded":"2020-03-19T21:18:30.007327+0000","last_fullsized":"2020-03-19T21:18:30.007327+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:13.185694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:13.185694+0000","last_clean_scrub_stamp":"2020-03-19T21:18:13.185694+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":"20","reported_epoch":"23","state":"active+clean","last_fresh":"2020-03-19T21:18:29.245896+0000","last_change":"2020-03-19T21:18:10.182075+0000","last_active":"2020-03-19T21:18:29.245896+0000","last_peered":"2020-03-19T21:18:29.245896+0000","last_clean":"2020-03-19T21:18:29.245896+0000","last_became_active":"2020-03-19T21:18:10.181738+0000","last_became_peered":"2020-03-19T21:18:10.181738+0000","last_unstale":"2020-03-19T21:18:29.245896+0000","last_undegraded":"2020-03-19T21:18:29.245896+0000","last_fullsized":"2020-03-19T21:18:29.245896+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:09.169195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:09.169195+0000","last_clean_scrub_stamp":"2020-03-19T21:18:09.169195+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"24'10","reported_seq":"28","reported_epoch":"24","state":"active+clean","last_fresh":"2020-03-19T21:18:32.435966+0000","last_change":"2020-03-19T21:18:13.197376+0000","last_active":"2020-03-19T21:18:32.435966+0000","last_peered":"2020-03-19T21:18:32.435966+0000","last_clean":"2020-03-19T21:18:32.435966+0000","last_became_active":"2020-03-19T21:18:13.196089+0000","last_became_peered":"2020-03-19T21:18:13.196089+0000","last_unstale":"2020-03-19T21:18:32.435966+0000","last_undegraded":"2020-03-19T21:18:32.435966+0000","last_fullsized":"2020-03-19T21:18:32.435966+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:12.182524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:12.182524+0000","last_clean_scrub_stamp":"2020-03-19T21:18:12.182524+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1958,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":4,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":8,"acting":8,"num_store_stats":3},{"poolid":3,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":2718,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":5436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738378,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":1048788,"kb_used_data":204,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":103808812,"statfs":{"total":107374182400,"available":106300223488,"internally_reserved":1073741824,"allocated":208896,"data_stored":77785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45400000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59699999999999998}]}]},{"osd":0,"up_from":8,"seq":34359738377,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":1048812,"kb_used_data":228,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":103808788,"statfs":{"total":107374182400,"available":106300198912,"internally_reserved":1073741824,"allocated":233472,"data_stored":80435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71599999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77100000000000002}]}]},{"osd":1,"up_from":8,"seq":34359738377,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":1048816,"kb_used_data":232,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":103808784,"statfs":{"total":107374182400,"available":106300194816,"internally_reserved":1073741824,"allocated":237568,"data_stored":80469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51400000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66300000000000003}]}]}]}} 2020-03-19T21:18:37.096 INFO:tasks.ceph:pgid 4.0 last_scrub_stamp 2020-03-19T21:18:23.204329+0000 time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=21, tm_min=18, tm_sec=23, tm_wday=3, tm_yday=79, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=21, tm_min=18, tm_sec=34, tm_wday=3, tm_yday=79, tm_isdst=0) 2020-03-19T21:18:37.096 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2020-03-19T21:18:13.185694+0000 time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=21, tm_min=18, tm_sec=13, tm_wday=3, tm_yday=79, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=21, tm_min=18, tm_sec=34, tm_wday=3, tm_yday=79, tm_isdst=0) 2020-03-19T21:18:37.097 INFO:tasks.ceph:pgid 1.5 last_scrub_stamp 2020-03-19T21:18:09.169195+0000 time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=21, tm_min=18, tm_sec=9, tm_wday=3, tm_yday=79, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=21, tm_min=18, tm_sec=34, tm_wday=3, tm_yday=79, tm_isdst=0) 2020-03-19T21:18:37.097 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2020-03-19T21:18:12.182524+0000 time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=21, tm_min=18, tm_sec=12, tm_wday=3, tm_yday=79, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=21, tm_min=18, tm_sec=34, tm_wday=3, tm_yday=79, tm_isdst=0) 2020-03-19T21:18:37.097 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2020-03-19T21:18:23.204329+0000 time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=21, tm_min=18, tm_sec=23, tm_wday=3, tm_yday=79, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=21, tm_min=18, tm_sec=34, tm_wday=3, tm_yday=79, tm_isdst=0) 2020-03-19T21:18:37.097 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2020-03-19T21:18:13.185694+0000 time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=21, tm_min=18, tm_sec=13, tm_wday=3, tm_yday=79, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=21, tm_min=18, tm_sec=34, tm_wday=3, tm_yday=79, tm_isdst=0) 2020-03-19T21:18:37.097 INFO:tasks.ceph:pgid 1.4 last_scrub_stamp 2020-03-19T21:18:09.169195+0000 time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=21, tm_min=18, tm_sec=9, tm_wday=3, tm_yday=79, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=21, tm_min=18, tm_sec=34, tm_wday=3, tm_yday=79, tm_isdst=0) 2020-03-19T21:18:37.098 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2020-03-19T21:18:12.182524+0000 time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=21, tm_min=18, tm_sec=12, tm_wday=3, tm_yday=79, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=21, tm_min=18, tm_sec=34, tm_wday=3, tm_yday=79, tm_isdst=0) 2020-03-19T21:18:37.098 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2020-03-19T21:18:13.185694+0000 time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=21, tm_min=18, tm_sec=13, tm_wday=3, tm_yday=79, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=21, tm_min=18, tm_sec=34, tm_wday=3, tm_yday=79, tm_isdst=0) 2020-03-19T21:18:37.098 INFO:tasks.ceph:pgid 1.3 last_scrub_stamp 2020-03-19T21:18:09.169195+0000 time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=21, tm_min=18, tm_sec=9, tm_wday=3, tm_yday=79, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=21, tm_min=18, tm_sec=34, tm_wday=3, tm_yday=79, tm_isdst=0) 2020-03-19T21:18:37.098 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2020-03-19T21:18:12.182524+0000 time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=21, tm_min=18, tm_sec=12, tm_wday=3, tm_yday=79, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=21, tm_min=18, tm_sec=34, tm_wday=3, tm_yday=79, tm_isdst=0) 2020-03-19T21:18:37.099 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2020-03-19T21:18:13.185694+0000 time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=21, tm_min=18, tm_sec=13, tm_wday=3, tm_yday=79, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=21, tm_min=18, tm_sec=34, tm_wday=3, tm_yday=79, tm_isdst=0) 2020-03-19T21:18:37.099 INFO:tasks.ceph:pgid 1.2 last_scrub_stamp 2020-03-19T21:18:09.169195+0000 time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=21, tm_min=18, tm_sec=9, tm_wday=3, tm_yday=79, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=21, tm_min=18, tm_sec=34, tm_wday=3, tm_yday=79, tm_isdst=0) 2020-03-19T21:18:37.099 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2020-03-19T21:18:12.182524+0000 time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=21, tm_min=18, tm_sec=12, tm_wday=3, tm_yday=79, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=21, tm_min=18, tm_sec=34, tm_wday=3, tm_yday=79, tm_isdst=0) 2020-03-19T21:18:37.099 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2020-03-19T21:18:13.185694+0000 time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=21, tm_min=18, tm_sec=13, tm_wday=3, tm_yday=79, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=21, tm_min=18, tm_sec=34, tm_wday=3, tm_yday=79, tm_isdst=0) 2020-03-19T21:18:37.101 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2020-03-19T21:18:09.169195+0000 time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=21, tm_min=18, tm_sec=9, tm_wday=3, tm_yday=79, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=21, tm_min=18, tm_sec=34, tm_wday=3, tm_yday=79, tm_isdst=0) 2020-03-19T21:18:37.101 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2020-03-19T21:18:12.182524+0000 time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=21, tm_min=18, tm_sec=12, tm_wday=3, tm_yday=79, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=21, tm_min=18, tm_sec=34, tm_wday=3, tm_yday=79, tm_isdst=0) 2020-03-19T21:18:37.101 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2020-03-19T21:18:13.185694+0000 time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=21, tm_min=18, tm_sec=13, tm_wday=3, tm_yday=79, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=21, tm_min=18, tm_sec=34, tm_wday=3, tm_yday=79, tm_isdst=0) 2020-03-19T21:18:37.101 INFO:tasks.ceph:pgid 1.1 last_scrub_stamp 2020-03-19T21:18:09.169195+0000 time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=21, tm_min=18, tm_sec=9, tm_wday=3, tm_yday=79, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=21, tm_min=18, tm_sec=34, tm_wday=3, tm_yday=79, tm_isdst=0) 2020-03-19T21:18:37.102 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2020-03-19T21:18:12.182524+0000 time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=21, tm_min=18, tm_sec=12, tm_wday=3, tm_yday=79, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=21, tm_min=18, tm_sec=34, tm_wday=3, tm_yday=79, tm_isdst=0) 2020-03-19T21:18:37.102 INFO:tasks.ceph:pgid 4.3 last_scrub_stamp 2020-03-19T21:18:23.204329+0000 time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=21, tm_min=18, tm_sec=23, tm_wday=3, tm_yday=79, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=21, tm_min=18, tm_sec=34, tm_wday=3, tm_yday=79, tm_isdst=0) 2020-03-19T21:18:37.103 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2020-03-19T21:18:13.185694+0000 time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=21, tm_min=18, tm_sec=13, tm_wday=3, tm_yday=79, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=21, tm_min=18, tm_sec=34, tm_wday=3, tm_yday=79, tm_isdst=0) 2020-03-19T21:18:37.103 INFO:tasks.ceph:pgid 1.6 last_scrub_stamp 2020-03-19T21:18:09.169195+0000 time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=21, tm_min=18, tm_sec=9, tm_wday=3, tm_yday=79, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=21, tm_min=18, tm_sec=34, tm_wday=3, tm_yday=79, tm_isdst=0) 2020-03-19T21:18:37.103 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2020-03-19T21:18:12.182524+0000 time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=21, tm_min=18, tm_sec=12, tm_wday=3, tm_yday=79, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=21, tm_min=18, tm_sec=34, tm_wday=3, tm_yday=79, tm_isdst=0) 2020-03-19T21:18:37.103 INFO:tasks.ceph:pgid 4.2 last_scrub_stamp 2020-03-19T21:18:23.204329+0000 time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=21, tm_min=18, tm_sec=23, tm_wday=3, tm_yday=79, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=21, tm_min=18, tm_sec=34, tm_wday=3, tm_yday=79, tm_isdst=0) 2020-03-19T21:18:37.103 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2020-03-19T21:18:13.185694+0000 time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=21, tm_min=18, tm_sec=13, tm_wday=3, tm_yday=79, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=21, tm_min=18, tm_sec=34, tm_wday=3, tm_yday=79, tm_isdst=0) 2020-03-19T21:18:37.103 INFO:tasks.ceph:pgid 1.7 last_scrub_stamp 2020-03-19T21:18:09.169195+0000 time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=21, tm_min=18, tm_sec=9, tm_wday=3, tm_yday=79, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=21, tm_min=18, tm_sec=34, tm_wday=3, tm_yday=79, tm_isdst=0) 2020-03-19T21:18:37.104 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2020-03-19T21:18:12.182524+0000 time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=21, tm_min=18, tm_sec=12, tm_wday=3, tm_yday=79, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=21, tm_min=18, tm_sec=34, tm_wday=3, tm_yday=79, tm_isdst=0) 2020-03-19T21:18:37.104 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2020-03-19T21:18:57.105 INFO:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early pg dump --format=json 2020-03-19T21:18:57.233 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:57.221+0000 7f9fcd278700 1 Processor -- start 2020-03-19T21:18:57.233 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:57.221+0000 7f9fcd278700 1 -- start start 2020-03-19T21:18:57.234 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:57.222+0000 7f9fcd278700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f9fc808d000 0x7f9fc80907c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T21:18:57.234 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:57.222+0000 7f9fcd278700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fc80605e0 con 0x7f9fc808d3e0 2020-03-19T21:18:57.234 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:57.222+0000 7f9fcd278700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fc808ef70 con 0x7f9fc808d000 2020-03-19T21:18:57.234 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:57.222+0000 7f9fc6ffd700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f9fc808d000 0x7f9fc80907c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:57.234 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:57.222+0000 7f9fc67fc700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f9fc808d3e0 0x7f9fc808d890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:54148/0 (socket says 172.21.15.170:54148) 2020-03-19T21:18:57.234 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:57.222+0000 7f9fc67fc700 1 -- 172.21.15.170:0/453680787 learned_addr learned my addr 172.21.15.170:0/453680787 (peer_addr_for_me v1:172.21.15.170:0/0) 2020-03-19T21:18:57.234 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:57.222+0000 7f9fc6ffd700 1 -- 172.21.15.170:0/453680787 >> v1:172.21.15.170:6789/0 conn(0x7f9fc808d3e0 legacy=0x7f9fc808d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:57.235 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:57.222+0000 7f9fc5ffb700 1 -- 172.21.15.170:0/453680787 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2546454954 0 0) 0x7f9fc80605e0 con 0x7f9fc808d3e0 2020-03-19T21:18:57.235 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:57.222+0000 7f9fc6ffd700 1 -- 172.21.15.170:0/453680787 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fc8060810 con 0x7f9fc808d000 2020-03-19T21:18:57.235 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:57.222+0000 7f9fc6ffd700 1 --2- 172.21.15.170:0/453680787 >> v2:172.21.15.170:3300/0 conn(0x7f9fc808d000 0x7f9fc80907c0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f9fb8002f30 tx=0x7f9fb8007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:57.235 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:57.222+0000 7f9fc6ffd700 1 --2- 172.21.15.170:0/453680787 >> v2:172.21.15.170:3300/0 conn(0x7f9fc808d000 0x7f9fc80907c0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f9fb8002f30 tx=0x7f9fb8007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:57.235 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:57.222+0000 7f9fc6ffd700 1 --2- 172.21.15.170:0/453680787 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9fc808d000 0x7f9fc80907c0 secure :-1 s=READY pgs=299 cs=0 l=1 rx=0x7f9fb8002f30 tx=0x7f9fb8007a40).ready entity=mon.0 client_cookie=49d40964d81a76e0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:57.235 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:57.222+0000 7f9fc6ffd700 1 --2- 172.21.15.170:0/453680787 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9fc808d000 0x7f9fc80907c0 secure :-1 s=THROTTLE_DONE pgs=299 cs=0 l=1 rx=0x7f9fb8002f30 tx=0x7f9fb8007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:57.236 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:57.222+0000 7f9fc5ffb700 1 -- 172.21.15.170:0/453680787 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f9fb80113f0 con 0x7f9fc808d000 2020-03-19T21:18:57.236 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:57.222+0000 7f9fc6ffd700 1 --2- 172.21.15.170:0/453680787 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9fc808d000 0x7f9fc80907c0 secure :-1 s=THROTTLE_DONE pgs=299 cs=0 l=1 rx=0x7f9fb8002f30 tx=0x7f9fb8007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:57.236 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:57.222+0000 7f9fc5ffb700 1 -- 172.21.15.170:0/453680787 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fb8011550 con 0x7f9fc808d000 2020-03-19T21:18:57.236 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:57.222+0000 7f9fc6ffd700 1 --2- 172.21.15.170:0/453680787 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9fc808d000 0x7f9fc80907c0 secure :-1 s=THROTTLE_DONE pgs=299 cs=0 l=1 rx=0x7f9fb8002f30 tx=0x7f9fb8007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:57.236 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:57.222+0000 7f9fc5ffb700 1 -- 172.21.15.170:0/453680787 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f9fb80117f0 con 0x7f9fc808d000 2020-03-19T21:18:57.236 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:57.222+0000 7f9fcd278700 1 -- 172.21.15.170:0/453680787 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9fc808d000 msgr2=0x7f9fc80907c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:57.237 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:57.222+0000 7f9fcd278700 1 --2- 172.21.15.170:0/453680787 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9fc808d000 0x7f9fc80907c0 secure :-1 s=READY pgs=299 cs=0 l=1 rx=0x7f9fb8002f30 tx=0x7f9fb8007a40).stop 2020-03-19T21:18:57.237 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:57.223+0000 7f9fcd278700 1 -- 172.21.15.170:0/453680787 shutdown_connections 2020-03-19T21:18:57.237 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:57.223+0000 7f9fcd278700 1 --2- 172.21.15.170:0/453680787 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9fc808d000 0x7f9fc80907c0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:57.237 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:57.223+0000 7f9fcd278700 1 -- 172.21.15.170:0/453680787 >> 172.21.15.170:0/453680787 conn(0x7f9fc8091540 msgr2=0x7f9fc8091950 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:57.237 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:57.223+0000 7f9fcd278700 1 -- 172.21.15.170:0/453680787 shutdown_connections 2020-03-19T21:18:57.237 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:57.223+0000 7f9fcd278700 1 -- 172.21.15.170:0/453680787 wait complete. 2020-03-19T21:18:57.237 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:57.224+0000 7f9fcd278700 1 Processor -- start 2020-03-19T21:18:57.238 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:57.224+0000 7f9fcd278700 1 -- start start 2020-03-19T21:18:57.238 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:57.224+0000 7f9fcd278700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f9fc808d000 0x7f9fc8122560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:57.238 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:57.224+0000 7f9fcd278700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fc8121350 con 0x7f9fc808d3e0 2020-03-19T21:18:57.238 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:57.224+0000 7f9fcd278700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fc811f010 con 0x7f9fc808d000 2020-03-19T21:18:57.238 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:57.224+0000 7f9fc6ffd700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f9fc808d000 0x7f9fc8122560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:57.238 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:57.224+0000 7f9fc67fc700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f9fc808d3e0 0x7f9fc81229c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:54154/0 (socket says 172.21.15.170:54154) 2020-03-19T21:18:57.238 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:57.224+0000 7f9fc6ffd700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f9fc808d000 0x7f9fc8122560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53388/0 (socket says 172.21.15.170:53388) 2020-03-19T21:18:57.239 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:57.224+0000 7f9fc67fc700 1 -- 172.21.15.170:0/3556999005 learned_addr learned my addr 172.21.15.170:0/3556999005 (peer_addr_for_me v1:172.21.15.170:0/0) 2020-03-19T21:18:57.239 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:57.225+0000 7f9fc6ffd700 1 -- 172.21.15.170:0/3556999005 >> v1:172.21.15.170:6789/0 conn(0x7f9fc808d3e0 legacy=0x7f9fc81229c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:57.239 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:57.225+0000 7f9fc6ffd700 1 -- 172.21.15.170:0/3556999005 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fc8121040 con 0x7f9fc808d000 2020-03-19T21:18:57.239 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:57.225+0000 7f9fb7fff700 1 -- 172.21.15.170:0/3556999005 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3756996459 0 0) 0x7f9fc8121350 con 0x7f9fc808d3e0 2020-03-19T21:18:57.239 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:57.225+0000 7f9fc6ffd700 1 --2- 172.21.15.170:0/3556999005 >> v2:172.21.15.170:3300/0 conn(0x7f9fc808d000 0x7f9fc8122560 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f9fb8002f30 tx=0x7f9fb8007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:57.239 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:57.225+0000 7f9fc6ffd700 1 --2- 172.21.15.170:0/3556999005 >> v2:172.21.15.170:3300/0 conn(0x7f9fc808d000 0x7f9fc8122560 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f9fb8002f30 tx=0x7f9fb8007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:57.239 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:57.225+0000 7f9fc6ffd700 1 --2- 172.21.15.170:0/3556999005 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9fc808d000 0x7f9fc8122560 secure :-1 s=READY pgs=301 cs=0 l=1 rx=0x7f9fb8002f30 tx=0x7f9fb8007a40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:57.240 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:57.225+0000 7f9fc6ffd700 1 --2- 172.21.15.170:0/3556999005 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9fc808d000 0x7f9fc8122560 secure :-1 s=THROTTLE_DONE pgs=301 cs=0 l=1 rx=0x7f9fb8002f30 tx=0x7f9fb8007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:57.240 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:57.225+0000 7f9fb7fff700 1 -- 172.21.15.170:0/3556999005 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f9fb800f7c0 con 0x7f9fc808d000 2020-03-19T21:18:57.240 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:57.225+0000 7f9fc6ffd700 1 --2- 172.21.15.170:0/3556999005 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9fc808d000 0x7f9fc8122560 secure :-1 s=THROTTLE_DONE pgs=301 cs=0 l=1 rx=0x7f9fb8002f30 tx=0x7f9fb8007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:57.240 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:57.225+0000 7f9fb7fff700 1 -- 172.21.15.170:0/3556999005 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fb8007150 con 0x7f9fc808d000 2020-03-19T21:18:57.240 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:57.225+0000 7f9fc6ffd700 1 --2- 172.21.15.170:0/3556999005 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9fc808d000 0x7f9fc8122560 secure :-1 s=THROTTLE_DONE pgs=301 cs=0 l=1 rx=0x7f9fb8002f30 tx=0x7f9fb8007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:57.240 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:57.225+0000 7f9fcd278700 1 -- 172.21.15.170:0/3556999005 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fc811eb90 con 0x7f9fc808d000 2020-03-19T21:18:57.241 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:57.225+0000 7f9fb7fff700 1 -- 172.21.15.170:0/3556999005 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f9fb80073f0 con 0x7f9fc808d000 2020-03-19T21:18:57.241 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:57.225+0000 7f9fcd278700 1 -- 172.21.15.170:0/3556999005 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fc811fe20 con 0x7f9fc808d000 2020-03-19T21:18:57.242 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:57.225+0000 7f9fc6ffd700 1 --2- 172.21.15.170:0/3556999005 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9fc808d000 0x7f9fc8122560 secure :-1 s=THROTTLE_DONE pgs=301 cs=0 l=1 rx=0x7f9fb8002f30 tx=0x7f9fb8007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:57.242 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:57.225+0000 7f9fb7fff700 1 -- 172.21.15.170:0/3556999005 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 4) v1 ==== 29603+0+0 (secure 0 0 0) 0x7f9fb80056c0 con 0x7f9fc808d000 2020-03-19T21:18:57.242 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:57.225+0000 7f9fc6ffd700 1 --2- 172.21.15.170:0/3556999005 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9fc808d000 0x7f9fc8122560 secure :-1 s=THROTTLE_DONE pgs=301 cs=0 l=1 rx=0x7f9fb8002f30 tx=0x7f9fb8007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:57.242 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:57.226+0000 7f9fb7fff700 1 --2- 172.21.15.170:0/3556999005 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f9fa4028200 0x7f9fa402a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:18:57.242 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:57.226+0000 7f9fb7fff700 1 -- 172.21.15.170:0/3556999005 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 3786+0+0 (secure 0 0 0) 0x7f9fb8034900 con 0x7f9fc808d000 2020-03-19T21:18:57.242 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:57.226+0000 7f9fc67fc700 1 --2- 172.21.15.170:0/3556999005 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f9fa4028200 0x7f9fa402a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:18:57.242 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:57.226+0000 7f9fcd278700 1 -- 172.21.15.170:0/3556999005 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fc8056db0 con 0x7f9fc808d000 2020-03-19T21:18:57.243 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:57.226+0000 7f9fc67fc700 1 --2- 172.21.15.170:0/3556999005 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f9fa4028200 0x7f9fa402a6c0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f9fbc005500 tx=0x7f9fbc0016f0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:57.243 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:57.226+0000 7f9fc67fc700 1 --2- 172.21.15.170:0/3556999005 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f9fa4028200 0x7f9fa402a6c0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f9fbc005500 tx=0x7f9fbc0016f0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:57.243 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:57.226+0000 7f9fc67fc700 1 --2- 172.21.15.170:0/3556999005 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f9fa4028200 0x7f9fa402a6c0 secure :-1 s=READY pgs=74 cs=0 l=1 rx=0x7f9fbc005500 tx=0x7f9fbc0016f0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:18:57.245 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:57.229+0000 7f9fc6ffd700 1 --2- 172.21.15.170:0/3556999005 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9fc808d000 0x7f9fc8122560 secure :-1 s=THROTTLE_DONE pgs=301 cs=0 l=1 rx=0x7f9fb8002f30 tx=0x7f9fb8007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:57.245 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:57.229+0000 7f9fb7fff700 1 -- 172.21.15.170:0/3556999005 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+126772 (secure 0 0 0) 0x7f9fb8009e30 con 0x7f9fc808d000 2020-03-19T21:18:57.364 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:57.362+0000 7f9fcd278700 1 -- 172.21.15.170:0/3556999005 --> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9fc806aaf0 con 0x7f9fa4028200 2020-03-19T21:18:57.368 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:57.365+0000 7f9fc67fc700 1 --2- 172.21.15.170:0/3556999005 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f9fa4028200 0x7f9fa402a6c0 secure :-1 s=THROTTLE_DONE pgs=74 cs=0 l=1 rx=0x7f9fbc005500 tx=0x7f9fbc0016f0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:18:57.368 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:57.365+0000 7f9fb7fff700 1 -- 172.21.15.170:0/3556999005 <== mgr.4102 v2:172.21.15.170:6824/20154 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+74073 (secure 0 0 0) 0x7f9fc806aaf0 con 0x7f9fa4028200 2020-03-19T21:18:57.368 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:18:57.369 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:57.366+0000 7f9fcd278700 1 -- 172.21.15.170:0/3556999005 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f9fa4028200 msgr2=0x7f9fa402a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:57.369 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:57.366+0000 7f9fcd278700 1 --2- 172.21.15.170:0/3556999005 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f9fa4028200 0x7f9fa402a6c0 secure :-1 s=READY pgs=74 cs=0 l=1 rx=0x7f9fbc005500 tx=0x7f9fbc0016f0).stop 2020-03-19T21:18:57.370 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:57.366+0000 7f9fcd278700 1 -- 172.21.15.170:0/3556999005 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9fc808d000 msgr2=0x7f9fc8122560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:18:57.370 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:57.366+0000 7f9fcd278700 1 --2- 172.21.15.170:0/3556999005 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9fc808d000 0x7f9fc8122560 secure :-1 s=READY pgs=301 cs=0 l=1 rx=0x7f9fb8002f30 tx=0x7f9fb8007a40).stop 2020-03-19T21:18:57.370 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:57.366+0000 7f9fcd278700 1 -- 172.21.15.170:0/3556999005 shutdown_connections 2020-03-19T21:18:57.370 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:57.366+0000 7f9fcd278700 1 --2- 172.21.15.170:0/3556999005 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f9fa4028200 0x7f9fa402a6c0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:57.370 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:57.366+0000 7f9fcd278700 1 --2- 172.21.15.170:0/3556999005 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9fc808d000 0x7f9fc8122560 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:18:57.370 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:57.366+0000 7f9fcd278700 1 -- 172.21.15.170:0/3556999005 >> 172.21.15.170:0/3556999005 conn(0x7f9fc8091540 msgr2=0x7f9fc80629b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:18:57.370 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:57.366+0000 7f9fcd278700 1 -- 172.21.15.170:0/3556999005 shutdown_connections 2020-03-19T21:18:57.371 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:18:57.366+0000 7f9fcd278700 1 -- 172.21.15.170:0/3556999005 wait complete. 2020-03-19T21:18:57.371 INFO:teuthology.orchestra.run.smithi170.stderr:dumped all 2020-03-19T21:18:57.379 INFO:teuthology.orchestra.run.smithi170.stdout:{"pg_ready":true,"pg_map":{"version":48,"stamp":"2020-03-19T21:18:56.517259+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2718,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":4620,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":53,"ondisk_log_size":53,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":314572800,"kb_used":3146416,"kb_used_data":664,"kb_used_omap":0,"kb_used_meta":3145728,"kb_avail":311426384,"statfs":{"total":322122547200,"available":318900617216,"internally_reserved":3221225472,"allocated":679936,"data_stored":238689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":3221225472},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":4620,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002705"},"pg_stats":[{"pgid":"4.0","version":"0'0","reported_seq":"16","reported_epoch":"24","state":"active+clean","last_fresh":"2020-03-19T21:18:47.372788+0000","last_change":"2020-03-19T21:18:47.372844+0000","last_active":"2020-03-19T21:18:47.372788+0000","last_peered":"2020-03-19T21:18:47.372788+0000","last_clean":"2020-03-19T21:18:47.372788+0000","last_became_active":"2020-03-19T21:18:24.224136+0000","last_became_peered":"2020-03-19T21:18:24.224136+0000","last_unstale":"2020-03-19T21:18:47.372788+0000","last_undegraded":"2020-03-19T21:18:47.372788+0000","last_fullsized":"2020-03-19T21:18:47.372788+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:47.372783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:47.372783+0000","last_clean_scrub_stamp":"2020-03-19T21:18:47.372783+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":"19","reported_epoch":"24","state":"active+clean","last_fresh":"2020-03-19T21:18:48.336985+0000","last_change":"2020-03-19T21:18:48.337038+0000","last_active":"2020-03-19T21:18:48.336985+0000","last_peered":"2020-03-19T21:18:48.336985+0000","last_clean":"2020-03-19T21:18:48.336985+0000","last_became_active":"2020-03-19T21:18:13.525041+0000","last_became_peered":"2020-03-19T21:18:13.525041+0000","last_unstale":"2020-03-19T21:18:48.336985+0000","last_undegraded":"2020-03-19T21:18:48.336985+0000","last_fullsized":"2020-03-19T21:18:48.336985+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:48.336979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:48.336979+0000","last_clean_scrub_stamp":"2020-03-19T21:18:48.336979+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":"23","reported_epoch":"24","state":"active+clean","last_fresh":"2020-03-19T21:18:41.280273+0000","last_change":"2020-03-19T21:18:41.280344+0000","last_active":"2020-03-19T21:18:41.280273+0000","last_peered":"2020-03-19T21:18:41.280273+0000","last_clean":"2020-03-19T21:18:41.280273+0000","last_became_active":"2020-03-19T21:18:10.186584+0000","last_became_peered":"2020-03-19T21:18:10.186584+0000","last_unstale":"2020-03-19T21:18:41.280273+0000","last_undegraded":"2020-03-19T21:18:41.280273+0000","last_fullsized":"2020-03-19T21:18:41.280273+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:41.280268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:41.280268+0000","last_clean_scrub_stamp":"2020-03-19T21:18:41.280268+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"24'6","reported_seq":"27","reported_epoch":"24","state":"active+clean","last_fresh":"2020-03-19T21:18:55.565763+0000","last_change":"2020-03-19T21:18:55.565821+0000","last_active":"2020-03-19T21:18:55.565763+0000","last_peered":"2020-03-19T21:18:55.565763+0000","last_clean":"2020-03-19T21:18:55.565763+0000","last_became_active":"2020-03-19T21:18:13.197671+0000","last_became_peered":"2020-03-19T21:18:13.197671+0000","last_unstale":"2020-03-19T21:18:55.565763+0000","last_undegraded":"2020-03-19T21:18:55.565763+0000","last_fullsized":"2020-03-19T21:18:55.565763+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"24'6","last_scrub_stamp":"2020-03-19T21:18:55.565757+0000","last_deep_scrub":"24'6","last_deep_scrub_stamp":"2020-03-19T21:18:55.565757+0000","last_clean_scrub_stamp":"2020-03-19T21:18:55.565757+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":"16","reported_epoch":"24","state":"active+clean","last_fresh":"2020-03-19T21:18:49.352476+0000","last_change":"2020-03-19T21:18:49.352526+0000","last_active":"2020-03-19T21:18:49.352476+0000","last_peered":"2020-03-19T21:18:49.352476+0000","last_clean":"2020-03-19T21:18:49.352476+0000","last_became_active":"2020-03-19T21:18:24.224025+0000","last_became_peered":"2020-03-19T21:18:24.224025+0000","last_unstale":"2020-03-19T21:18:49.352476+0000","last_undegraded":"2020-03-19T21:18:49.352476+0000","last_fullsized":"2020-03-19T21:18:49.352476+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:49.352468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:49.352468+0000","last_clean_scrub_stamp":"2020-03-19T21:18:49.352468+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":"19","reported_epoch":"24","state":"active+clean","last_fresh":"2020-03-19T21:18:45.305647+0000","last_change":"2020-03-19T21:18:45.305700+0000","last_active":"2020-03-19T21:18:45.305647+0000","last_peered":"2020-03-19T21:18:45.305647+0000","last_clean":"2020-03-19T21:18:45.305647+0000","last_became_active":"2020-03-19T21:18:13.524563+0000","last_became_peered":"2020-03-19T21:18:13.524563+0000","last_unstale":"2020-03-19T21:18:45.305647+0000","last_undegraded":"2020-03-19T21:18:45.305647+0000","last_fullsized":"2020-03-19T21:18:45.305647+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:45.305641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:45.305641+0000","last_clean_scrub_stamp":"2020-03-19T21:18:45.305641+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":"23","reported_epoch":"24","state":"active+clean","last_fresh":"2020-03-19T21:18:41.401276+0000","last_change":"2020-03-19T21:18:41.401331+0000","last_active":"2020-03-19T21:18:41.401276+0000","last_peered":"2020-03-19T21:18:41.401276+0000","last_clean":"2020-03-19T21:18:41.401276+0000","last_became_active":"2020-03-19T21:18:10.183810+0000","last_became_peered":"2020-03-19T21:18:10.183810+0000","last_unstale":"2020-03-19T21:18:41.401276+0000","last_undegraded":"2020-03-19T21:18:41.401276+0000","last_fullsized":"2020-03-19T21:18:41.401276+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:41.401271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:41.401271+0000","last_clean_scrub_stamp":"2020-03-19T21:18:41.401271+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"16'7","reported_seq":"25","reported_epoch":"23","state":"active+clean","last_fresh":"2020-03-19T21:18:29.245983+0000","last_change":"2020-03-19T21:18:13.196212+0000","last_active":"2020-03-19T21:18:29.245983+0000","last_peered":"2020-03-19T21:18:29.245983+0000","last_clean":"2020-03-19T21:18:29.245983+0000","last_became_active":"2020-03-19T21:18:13.195896+0000","last_became_peered":"2020-03-19T21:18:13.195896+0000","last_unstale":"2020-03-19T21:18:29.245983+0000","last_undegraded":"2020-03-19T21:18:29.245983+0000","last_fullsized":"2020-03-19T21:18:29.245983+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:12.182524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:12.182524+0000","last_clean_scrub_stamp":"2020-03-19T21:18:12.182524+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":566,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":"19","reported_epoch":"24","state":"active+clean","last_fresh":"2020-03-19T21:18:41.293088+0000","last_change":"2020-03-19T21:18:41.293146+0000","last_active":"2020-03-19T21:18:41.293088+0000","last_peered":"2020-03-19T21:18:41.293088+0000","last_clean":"2020-03-19T21:18:41.293088+0000","last_became_active":"2020-03-19T21:18:13.523806+0000","last_became_peered":"2020-03-19T21:18:13.523806+0000","last_unstale":"2020-03-19T21:18:41.293088+0000","last_undegraded":"2020-03-19T21:18:41.293088+0000","last_fullsized":"2020-03-19T21:18:41.293088+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:41.293083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:41.293083+0000","last_clean_scrub_stamp":"2020-03-19T21:18:41.293083+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":"23","reported_epoch":"24","state":"active+clean","last_fresh":"2020-03-19T21:18:39.363113+0000","last_change":"2020-03-19T21:18:39.363190+0000","last_active":"2020-03-19T21:18:39.363113+0000","last_peered":"2020-03-19T21:18:39.363113+0000","last_clean":"2020-03-19T21:18:39.363113+0000","last_became_active":"2020-03-19T21:18:10.183788+0000","last_became_peered":"2020-03-19T21:18:10.183788+0000","last_unstale":"2020-03-19T21:18:39.363113+0000","last_undegraded":"2020-03-19T21:18:39.363113+0000","last_fullsized":"2020-03-19T21:18:39.363113+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:39.363109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:39.363109+0000","last_clean_scrub_stamp":"2020-03-19T21:18:39.363109+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"16'8","reported_seq":"28","reported_epoch":"24","state":"active+clean","last_fresh":"2020-03-19T21:18:42.318384+0000","last_change":"2020-03-19T21:18:42.318461+0000","last_active":"2020-03-19T21:18:42.318384+0000","last_peered":"2020-03-19T21:18:42.318384+0000","last_clean":"2020-03-19T21:18:42.318384+0000","last_became_active":"2020-03-19T21:18:13.196823+0000","last_became_peered":"2020-03-19T21:18:13.196823+0000","last_unstale":"2020-03-19T21:18:42.318384+0000","last_undegraded":"2020-03-19T21:18:42.318384+0000","last_fullsized":"2020-03-19T21:18:42.318384+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"16'8","last_scrub_stamp":"2020-03-19T21:18:42.318376+0000","last_deep_scrub":"16'8","last_deep_scrub_stamp":"2020-03-19T21:18:42.318376+0000","last_clean_scrub_stamp":"2020-03-19T21:18:42.318376+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":"17","reported_epoch":"23","state":"active+clean","last_fresh":"2020-03-19T21:18:29.245702+0000","last_change":"2020-03-19T21:18:13.528239+0000","last_active":"2020-03-19T21:18:29.245702+0000","last_peered":"2020-03-19T21:18:29.245702+0000","last_clean":"2020-03-19T21:18:29.245702+0000","last_became_active":"2020-03-19T21:18:13.527916+0000","last_became_peered":"2020-03-19T21:18:13.527916+0000","last_unstale":"2020-03-19T21:18:29.245702+0000","last_undegraded":"2020-03-19T21:18:29.245702+0000","last_fullsized":"2020-03-19T21:18:29.245702+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:13.185694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:13.185694+0000","last_clean_scrub_stamp":"2020-03-19T21:18:13.185694+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":"22","reported_epoch":"24","state":"active+clean","last_fresh":"2020-03-19T21:18:38.275259+0000","last_change":"2020-03-19T21:18:38.275323+0000","last_active":"2020-03-19T21:18:38.275259+0000","last_peered":"2020-03-19T21:18:38.275259+0000","last_clean":"2020-03-19T21:18:38.275259+0000","last_became_active":"2020-03-19T21:18:10.187570+0000","last_became_peered":"2020-03-19T21:18:10.187570+0000","last_unstale":"2020-03-19T21:18:38.275259+0000","last_undegraded":"2020-03-19T21:18:38.275259+0000","last_fullsized":"2020-03-19T21:18:38.275259+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:38.275254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:38.275254+0000","last_clean_scrub_stamp":"2020-03-19T21:18:38.275254+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"20","reported_epoch":"24","state":"active+clean","last_fresh":"2020-03-19T21:18:43.343893+0000","last_change":"2020-03-19T21:18:43.343958+0000","last_active":"2020-03-19T21:18:43.343893+0000","last_peered":"2020-03-19T21:18:43.343893+0000","last_clean":"2020-03-19T21:18:43.343893+0000","last_became_active":"2020-03-19T21:18:13.197224+0000","last_became_peered":"2020-03-19T21:18:13.197224+0000","last_unstale":"2020-03-19T21:18:43.343893+0000","last_undegraded":"2020-03-19T21:18:43.343893+0000","last_fullsized":"2020-03-19T21:18:43.343893+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:43.343884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:43.343884+0000","last_clean_scrub_stamp":"2020-03-19T21:18:43.343884+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":"17","reported_epoch":"23","state":"active+clean","last_fresh":"2020-03-19T21:18:29.245873+0000","last_change":"2020-03-19T21:18:13.524218+0000","last_active":"2020-03-19T21:18:29.245873+0000","last_peered":"2020-03-19T21:18:29.245873+0000","last_clean":"2020-03-19T21:18:29.245873+0000","last_became_active":"2020-03-19T21:18:13.523902+0000","last_became_peered":"2020-03-19T21:18:13.523902+0000","last_unstale":"2020-03-19T21:18:29.245873+0000","last_undegraded":"2020-03-19T21:18:29.245873+0000","last_fullsized":"2020-03-19T21:18:29.245873+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:13.185694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:13.185694+0000","last_clean_scrub_stamp":"2020-03-19T21:18:13.185694+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":"23","reported_epoch":"24","state":"active+clean","last_fresh":"2020-03-19T21:18:38.360368+0000","last_change":"2020-03-19T21:18:38.360418+0000","last_active":"2020-03-19T21:18:38.360368+0000","last_peered":"2020-03-19T21:18:38.360368+0000","last_clean":"2020-03-19T21:18:38.360368+0000","last_became_active":"2020-03-19T21:18:10.184433+0000","last_became_peered":"2020-03-19T21:18:10.184433+0000","last_unstale":"2020-03-19T21:18:38.360368+0000","last_undegraded":"2020-03-19T21:18:38.360368+0000","last_fullsized":"2020-03-19T21:18:38.360368+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:38.360362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:38.360362+0000","last_clean_scrub_stamp":"2020-03-19T21:18:38.360362+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"24'8","reported_seq":"29","reported_epoch":"24","state":"active+clean","last_fresh":"2020-03-19T21:18:47.489117+0000","last_change":"2020-03-19T21:18:47.489175+0000","last_active":"2020-03-19T21:18:47.489117+0000","last_peered":"2020-03-19T21:18:47.489117+0000","last_clean":"2020-03-19T21:18:47.489117+0000","last_became_active":"2020-03-19T21:18:13.194849+0000","last_became_peered":"2020-03-19T21:18:13.194849+0000","last_unstale":"2020-03-19T21:18:47.489117+0000","last_undegraded":"2020-03-19T21:18:47.489117+0000","last_fullsized":"2020-03-19T21:18:47.489117+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"24'8","last_scrub_stamp":"2020-03-19T21:18:47.489109+0000","last_deep_scrub":"24'8","last_deep_scrub_stamp":"2020-03-19T21:18:47.489109+0000","last_clean_scrub_stamp":"2020-03-19T21:18:47.489109+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":4620,"num_omap_keys":10,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":"19","reported_epoch":"24","state":"active+clean","last_fresh":"2020-03-19T21:18:42.275966+0000","last_change":"2020-03-19T21:18:42.276022+0000","last_active":"2020-03-19T21:18:42.275966+0000","last_peered":"2020-03-19T21:18:42.275966+0000","last_clean":"2020-03-19T21:18:42.275966+0000","last_became_active":"2020-03-19T21:18:13.522937+0000","last_became_peered":"2020-03-19T21:18:13.522937+0000","last_unstale":"2020-03-19T21:18:42.275966+0000","last_undegraded":"2020-03-19T21:18:42.275966+0000","last_fullsized":"2020-03-19T21:18:42.275966+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:42.275962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:42.275962+0000","last_clean_scrub_stamp":"2020-03-19T21:18:42.275962+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":"23","reported_epoch":"24","state":"active+clean","last_fresh":"2020-03-19T21:18:39.239015+0000","last_change":"2020-03-19T21:18:39.239071+0000","last_active":"2020-03-19T21:18:39.239015+0000","last_peered":"2020-03-19T21:18:39.239015+0000","last_clean":"2020-03-19T21:18:39.239015+0000","last_became_active":"2020-03-19T21:18:10.183404+0000","last_became_peered":"2020-03-19T21:18:10.183404+0000","last_unstale":"2020-03-19T21:18:39.239015+0000","last_undegraded":"2020-03-19T21:18:39.239015+0000","last_fullsized":"2020-03-19T21:18:39.239015+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:39.239009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:39.239009+0000","last_clean_scrub_stamp":"2020-03-19T21:18:39.239009+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"16'1","reported_seq":"21","reported_epoch":"24","state":"active+clean","last_fresh":"2020-03-19T21:18:40.270522+0000","last_change":"2020-03-19T21:18:40.270588+0000","last_active":"2020-03-19T21:18:40.270522+0000","last_peered":"2020-03-19T21:18:40.270522+0000","last_clean":"2020-03-19T21:18:40.270522+0000","last_became_active":"2020-03-19T21:18:13.198498+0000","last_became_peered":"2020-03-19T21:18:13.198498+0000","last_unstale":"2020-03-19T21:18:40.270522+0000","last_undegraded":"2020-03-19T21:18:40.270522+0000","last_fullsized":"2020-03-19T21:18:40.270522+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"16'1","last_scrub_stamp":"2020-03-19T21:18:40.270516+0000","last_deep_scrub":"16'1","last_deep_scrub_stamp":"2020-03-19T21:18:40.270516+0000","last_clean_scrub_stamp":"2020-03-19T21:18:40.270516+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":"16","reported_epoch":"24","state":"active+clean","last_fresh":"2020-03-19T21:18:49.332432+0000","last_change":"2020-03-19T21:18:49.332513+0000","last_active":"2020-03-19T21:18:49.332432+0000","last_peered":"2020-03-19T21:18:49.332432+0000","last_clean":"2020-03-19T21:18:49.332432+0000","last_became_active":"2020-03-19T21:18:24.224328+0000","last_became_peered":"2020-03-19T21:18:24.224328+0000","last_unstale":"2020-03-19T21:18:49.332432+0000","last_undegraded":"2020-03-19T21:18:49.332432+0000","last_fullsized":"2020-03-19T21:18:49.332432+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:49.332427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:49.332427+0000","last_clean_scrub_stamp":"2020-03-19T21:18:49.332427+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":"17","reported_epoch":"23","state":"active+clean","last_fresh":"2020-03-19T21:18:29.245491+0000","last_change":"2020-03-19T21:18:13.524754+0000","last_active":"2020-03-19T21:18:29.245491+0000","last_peered":"2020-03-19T21:18:29.245491+0000","last_clean":"2020-03-19T21:18:29.245491+0000","last_became_active":"2020-03-19T21:18:13.524437+0000","last_became_peered":"2020-03-19T21:18:13.524437+0000","last_unstale":"2020-03-19T21:18:29.245491+0000","last_undegraded":"2020-03-19T21:18:29.245491+0000","last_fullsized":"2020-03-19T21:18:29.245491+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:13.185694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:13.185694+0000","last_clean_scrub_stamp":"2020-03-19T21:18:13.185694+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":"23","reported_epoch":"24","state":"active+clean","last_fresh":"2020-03-19T21:18:43.440965+0000","last_change":"2020-03-19T21:18:43.441016+0000","last_active":"2020-03-19T21:18:43.440965+0000","last_peered":"2020-03-19T21:18:43.440965+0000","last_clean":"2020-03-19T21:18:43.440965+0000","last_became_active":"2020-03-19T21:18:10.184549+0000","last_became_peered":"2020-03-19T21:18:10.184549+0000","last_unstale":"2020-03-19T21:18:43.440965+0000","last_undegraded":"2020-03-19T21:18:43.440965+0000","last_fullsized":"2020-03-19T21:18:43.440965+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:43.440960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:43.440960+0000","last_clean_scrub_stamp":"2020-03-19T21:18:43.440960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"24'10","reported_seq":"31","reported_epoch":"24","state":"active+clean","last_fresh":"2020-03-19T21:18:52.575209+0000","last_change":"2020-03-19T21:18:52.575264+0000","last_active":"2020-03-19T21:18:52.575209+0000","last_peered":"2020-03-19T21:18:52.575209+0000","last_clean":"2020-03-19T21:18:52.575209+0000","last_became_active":"2020-03-19T21:18:13.195947+0000","last_became_peered":"2020-03-19T21:18:13.195947+0000","last_unstale":"2020-03-19T21:18:52.575209+0000","last_undegraded":"2020-03-19T21:18:52.575209+0000","last_fullsized":"2020-03-19T21:18:52.575209+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"24'10","last_scrub_stamp":"2020-03-19T21:18:52.575203+0000","last_deep_scrub":"24'10","last_deep_scrub_stamp":"2020-03-19T21:18:52.575203+0000","last_clean_scrub_stamp":"2020-03-19T21:18:52.575203+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":160,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":"14","reported_epoch":"23","state":"active+clean","last_fresh":"2020-03-19T21:18:29.245638+0000","last_change":"2020-03-19T21:18:24.223995+0000","last_active":"2020-03-19T21:18:29.245638+0000","last_peered":"2020-03-19T21:18:29.245638+0000","last_clean":"2020-03-19T21:18:29.245638+0000","last_became_active":"2020-03-19T21:18:24.223594+0000","last_became_peered":"2020-03-19T21:18:24.223594+0000","last_unstale":"2020-03-19T21:18:29.245638+0000","last_undegraded":"2020-03-19T21:18:29.245638+0000","last_fullsized":"2020-03-19T21:18:29.245638+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:23.204329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:23.204329+0000","last_clean_scrub_stamp":"2020-03-19T21:18:23.204329+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":"19","reported_epoch":"24","state":"active+clean","last_fresh":"2020-03-19T21:18:45.371339+0000","last_change":"2020-03-19T21:18:45.371397+0000","last_active":"2020-03-19T21:18:45.371339+0000","last_peered":"2020-03-19T21:18:45.371339+0000","last_clean":"2020-03-19T21:18:45.371339+0000","last_became_active":"2020-03-19T21:18:13.524938+0000","last_became_peered":"2020-03-19T21:18:13.524938+0000","last_unstale":"2020-03-19T21:18:45.371339+0000","last_undegraded":"2020-03-19T21:18:45.371339+0000","last_fullsized":"2020-03-19T21:18:45.371339+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:45.371335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:45.371335+0000","last_clean_scrub_stamp":"2020-03-19T21:18:45.371335+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":"23","reported_epoch":"24","state":"active+clean","last_fresh":"2020-03-19T21:18:45.446348+0000","last_change":"2020-03-19T21:18:45.446415+0000","last_active":"2020-03-19T21:18:45.446348+0000","last_peered":"2020-03-19T21:18:45.446348+0000","last_clean":"2020-03-19T21:18:45.446348+0000","last_became_active":"2020-03-19T21:18:10.181738+0000","last_became_peered":"2020-03-19T21:18:10.181738+0000","last_unstale":"2020-03-19T21:18:45.446348+0000","last_undegraded":"2020-03-19T21:18:45.446348+0000","last_fullsized":"2020-03-19T21:18:45.446348+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:45.446342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:45.446342+0000","last_clean_scrub_stamp":"2020-03-19T21:18:45.446342+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"24'13","reported_seq":"34","reported_epoch":"24","state":"active+clean","last_fresh":"2020-03-19T21:18:49.510676+0000","last_change":"2020-03-19T21:18:49.510739+0000","last_active":"2020-03-19T21:18:49.510676+0000","last_peered":"2020-03-19T21:18:49.510676+0000","last_clean":"2020-03-19T21:18:49.510676+0000","last_became_active":"2020-03-19T21:18:13.196089+0000","last_became_peered":"2020-03-19T21:18:13.196089+0000","last_unstale":"2020-03-19T21:18:49.510676+0000","last_undegraded":"2020-03-19T21:18:49.510676+0000","last_fullsized":"2020-03-19T21:18:49.510676+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"24'13","last_scrub_stamp":"2020-03-19T21:18:49.510667+0000","last_deep_scrub":"24'13","last_deep_scrub_stamp":"2020-03-19T21:18:49.510667+0000","last_clean_scrub_stamp":"2020-03-19T21:18:49.510667+0000","log_size":13,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1958,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":4,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":8,"acting":8,"num_store_stats":3},{"poolid":3,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":2718,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":4620,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":5436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":53,"ondisk_log_size":53,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738382,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":1048788,"kb_used_data":204,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":103808812,"statfs":{"total":107374182400,"available":106300223488,"internally_reserved":1073741824,"allocated":208896,"data_stored":77785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63400000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75}]}]},{"osd":0,"up_from":8,"seq":34359738381,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":1048812,"kb_used_data":228,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":103808788,"statfs":{"total":107374182400,"available":106300198912,"internally_reserved":1073741824,"allocated":233472,"data_stored":80435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67600000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58899999999999997}]}]},{"osd":1,"up_from":8,"seq":34359738381,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":1048816,"kb_used_data":232,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":103808784,"statfs":{"total":107374182400,"available":106300194816,"internally_reserved":1073741824,"allocated":237568,"data_stored":80469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47899999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65300000000000002}]}]}]}} 2020-03-19T21:18:57.382 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2020-03-19T21:18:12.182524+0000 time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=21, tm_min=18, tm_sec=12, tm_wday=3, tm_yday=79, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=21, tm_min=18, tm_sec=34, tm_wday=3, tm_yday=79, tm_isdst=0) 2020-03-19T21:18:57.382 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2020-03-19T21:18:13.185694+0000 time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=21, tm_min=18, tm_sec=13, tm_wday=3, tm_yday=79, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=21, tm_min=18, tm_sec=34, tm_wday=3, tm_yday=79, tm_isdst=0) 2020-03-19T21:18:57.383 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2020-03-19T21:18:13.185694+0000 time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=21, tm_min=18, tm_sec=13, tm_wday=3, tm_yday=79, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=21, tm_min=18, tm_sec=34, tm_wday=3, tm_yday=79, tm_isdst=0) 2020-03-19T21:18:57.383 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2020-03-19T21:18:13.185694+0000 time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=21, tm_min=18, tm_sec=13, tm_wday=3, tm_yday=79, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=21, tm_min=18, tm_sec=34, tm_wday=3, tm_yday=79, tm_isdst=0) 2020-03-19T21:18:57.383 INFO:tasks.ceph:pgid 4.2 last_scrub_stamp 2020-03-19T21:18:23.204329+0000 time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=21, tm_min=18, tm_sec=23, tm_wday=3, tm_yday=79, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=21, tm_min=18, tm_sec=34, tm_wday=3, tm_yday=79, tm_isdst=0) 2020-03-19T21:18:57.383 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2020-03-19T21:19:17.384 INFO:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early pg dump --format=json 2020-03-19T21:19:17.501 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.499+0000 7fa9486be700 1 Processor -- start 2020-03-19T21:19:17.501 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.500+0000 7fa9486be700 1 -- start start 2020-03-19T21:19:17.506 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.500+0000 7fa9486be700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fa94008d000 0x7fa9400907c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T21:19:17.506 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.500+0000 7fa9486be700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9400605e0 con 0x7fa94008d3e0 2020-03-19T21:19:17.506 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.500+0000 7fa9486be700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa94008ef70 con 0x7fa94008d000 2020-03-19T21:19:17.507 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.500+0000 7fa94645a700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fa94008d000 0x7fa9400907c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:19:17.507 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.500+0000 7fa945c59700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fa94008d3e0 0x7fa94008d890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:54162/0 (socket says 172.21.15.170:54162) 2020-03-19T21:19:17.507 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.500+0000 7fa94645a700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fa94008d000 0x7fa9400907c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53398/0 (socket says 172.21.15.170:53398) 2020-03-19T21:19:17.507 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.500+0000 7fa945c59700 1 -- 172.21.15.170:0/2868266543 learned_addr learned my addr 172.21.15.170:0/2868266543 (peer_addr_for_me v1:172.21.15.170:0/0) 2020-03-19T21:19:17.507 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.501+0000 7fa94645a700 1 -- 172.21.15.170:0/2868266543 >> v1:172.21.15.170:6789/0 conn(0x7fa94008d3e0 legacy=0x7fa94008d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:19:17.507 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.501+0000 7fa94645a700 1 -- 172.21.15.170:0/2868266543 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa940060810 con 0x7fa94008d000 2020-03-19T21:19:17.507 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.501+0000 7fa945458700 1 -- 172.21.15.170:0/2868266543 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 685559516 0 0) 0x7fa9400605e0 con 0x7fa94008d3e0 2020-03-19T21:19:17.508 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.501+0000 7fa94645a700 1 --2- 172.21.15.170:0/2868266543 >> v2:172.21.15.170:3300/0 conn(0x7fa94008d000 0x7fa9400907c0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7fa934002f30 tx=0x7fa934007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:19:17.508 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.501+0000 7fa94645a700 1 --2- 172.21.15.170:0/2868266543 >> v2:172.21.15.170:3300/0 conn(0x7fa94008d000 0x7fa9400907c0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7fa934002f30 tx=0x7fa934007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:19:17.508 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.501+0000 7fa94645a700 1 --2- 172.21.15.170:0/2868266543 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa94008d000 0x7fa9400907c0 secure :-1 s=READY pgs=303 cs=0 l=1 rx=0x7fa934002f30 tx=0x7fa934007a40).ready entity=mon.0 client_cookie=4976f42c1b566aac server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:19:17.508 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.501+0000 7fa94645a700 1 --2- 172.21.15.170:0/2868266543 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa94008d000 0x7fa9400907c0 secure :-1 s=THROTTLE_DONE pgs=303 cs=0 l=1 rx=0x7fa934002f30 tx=0x7fa934007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:19:17.508 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.501+0000 7fa945458700 1 -- 172.21.15.170:0/2868266543 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fa9340113f0 con 0x7fa94008d000 2020-03-19T21:19:17.508 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.501+0000 7fa94645a700 1 --2- 172.21.15.170:0/2868266543 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa94008d000 0x7fa9400907c0 secure :-1 s=THROTTLE_DONE pgs=303 cs=0 l=1 rx=0x7fa934002f30 tx=0x7fa934007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:19:17.508 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.501+0000 7fa945458700 1 -- 172.21.15.170:0/2868266543 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa934011550 con 0x7fa94008d000 2020-03-19T21:19:17.508 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.501+0000 7fa94645a700 1 --2- 172.21.15.170:0/2868266543 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa94008d000 0x7fa9400907c0 secure :-1 s=THROTTLE_DONE pgs=303 cs=0 l=1 rx=0x7fa934002f30 tx=0x7fa934007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:19:17.509 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.501+0000 7fa945458700 1 -- 172.21.15.170:0/2868266543 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fa9340117f0 con 0x7fa94008d000 2020-03-19T21:19:17.509 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.501+0000 7fa9486be700 1 -- 172.21.15.170:0/2868266543 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa94008d000 msgr2=0x7fa9400907c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:19:17.509 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.501+0000 7fa9486be700 1 --2- 172.21.15.170:0/2868266543 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa94008d000 0x7fa9400907c0 secure :-1 s=READY pgs=303 cs=0 l=1 rx=0x7fa934002f30 tx=0x7fa934007a40).stop 2020-03-19T21:19:17.509 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.501+0000 7fa9486be700 1 -- 172.21.15.170:0/2868266543 shutdown_connections 2020-03-19T21:19:17.509 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.501+0000 7fa9486be700 1 --2- 172.21.15.170:0/2868266543 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa94008d000 0x7fa9400907c0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:19:17.509 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.501+0000 7fa9486be700 1 -- 172.21.15.170:0/2868266543 >> 172.21.15.170:0/2868266543 conn(0x7fa940091540 msgr2=0x7fa940091950 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:19:17.510 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.501+0000 7fa9486be700 1 -- 172.21.15.170:0/2868266543 shutdown_connections 2020-03-19T21:19:17.510 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.501+0000 7fa9486be700 1 -- 172.21.15.170:0/2868266543 wait complete. 2020-03-19T21:19:17.510 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.502+0000 7fa9486be700 1 Processor -- start 2020-03-19T21:19:17.510 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.502+0000 7fa9486be700 1 -- start start 2020-03-19T21:19:17.511 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.503+0000 7fa9486be700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fa94008d3e0 0x7fa940066460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:19:17.511 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.503+0000 7fa9486be700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa94011e9b0 con 0x7fa94008d000 2020-03-19T21:19:17.511 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.503+0000 7fa9486be700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa94011f200 con 0x7fa94008d3e0 2020-03-19T21:19:17.511 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.503+0000 7fa94645a700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fa94008d000 0x7fa940063f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:54166/0 (socket says 172.21.15.170:54166) 2020-03-19T21:19:17.511 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.503+0000 7fa94645a700 1 -- 172.21.15.170:0/162160517 learned_addr learned my addr 172.21.15.170:0/162160517 (peer_addr_for_me v1:172.21.15.170:0/0) 2020-03-19T21:19:17.511 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.503+0000 7fa945c59700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fa94008d3e0 0x7fa940066460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:19:17.511 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.503+0000 7fa9337fe700 1 -- 172.21.15.170:0/162160517 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3398679061 0 0) 0x7fa94011e9b0 con 0x7fa94008d000 2020-03-19T21:19:17.512 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.503+0000 7fa9337fe700 1 -- 172.21.15.170:0/162160517 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa920002fe0 con 0x7fa94008d000 2020-03-19T21:19:17.512 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.503+0000 7fa945c59700 1 -- 172.21.15.170:0/162160517 >> v1:172.21.15.170:6789/0 conn(0x7fa94008d000 legacy=0x7fa940063f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:19:17.512 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.503+0000 7fa945c59700 1 -- 172.21.15.170:0/162160517 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa940065100 con 0x7fa94008d3e0 2020-03-19T21:19:17.512 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.503+0000 7fa945c59700 1 --2- 172.21.15.170:0/162160517 >> v2:172.21.15.170:3300/0 conn(0x7fa94008d3e0 0x7fa940066460 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7fa93c000b80 tx=0x7fa93c010860).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:19:17.512 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.503+0000 7fa945c59700 1 --2- 172.21.15.170:0/162160517 >> v2:172.21.15.170:3300/0 conn(0x7fa94008d3e0 0x7fa940066460 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7fa93c000b80 tx=0x7fa93c010860).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:19:17.512 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.503+0000 7fa945c59700 1 --2- 172.21.15.170:0/162160517 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa94008d3e0 0x7fa940066460 secure :-1 s=READY pgs=305 cs=0 l=1 rx=0x7fa93c000b80 tx=0x7fa93c010860).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:19:17.512 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.503+0000 7fa945c59700 1 --2- 172.21.15.170:0/162160517 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa94008d3e0 0x7fa940066460 secure :-1 s=THROTTLE_DONE pgs=305 cs=0 l=1 rx=0x7fa93c000b80 tx=0x7fa93c010860).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:19:17.513 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.503+0000 7fa9337fe700 1 -- 172.21.15.170:0/162160517 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fa93c005b20 con 0x7fa94008d3e0 2020-03-19T21:19:17.513 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.503+0000 7fa945c59700 1 --2- 172.21.15.170:0/162160517 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa94008d3e0 0x7fa940066460 secure :-1 s=THROTTLE_DONE pgs=305 cs=0 l=1 rx=0x7fa93c000b80 tx=0x7fa93c010860).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:19:17.513 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.503+0000 7fa945c59700 1 --2- 172.21.15.170:0/162160517 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa94008d3e0 0x7fa940066460 secure :-1 s=THROTTLE_DONE pgs=305 cs=0 l=1 rx=0x7fa93c000b80 tx=0x7fa93c010860).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:19:17.513 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.503+0000 7fa9337fe700 1 -- 172.21.15.170:0/162160517 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa93c005c80 con 0x7fa94008d3e0 2020-03-19T21:19:17.513 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.503+0000 7fa9337fe700 1 -- 172.21.15.170:0/162160517 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fa93c0060c0 con 0x7fa94008d3e0 2020-03-19T21:19:17.513 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.503+0000 7fa9486be700 1 -- 172.21.15.170:0/162160517 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa94011eef0 con 0x7fa94008d3e0 2020-03-19T21:19:17.513 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.503+0000 7fa9486be700 1 -- 172.21.15.170:0/162160517 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9400636f0 con 0x7fa94008d3e0 2020-03-19T21:19:17.514 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.504+0000 7fa945c59700 1 --2- 172.21.15.170:0/162160517 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa94008d3e0 0x7fa940066460 secure :-1 s=THROTTLE_DONE pgs=305 cs=0 l=1 rx=0x7fa93c000b80 tx=0x7fa93c010860).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:19:17.514 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.504+0000 7fa945c59700 1 --2- 172.21.15.170:0/162160517 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa94008d3e0 0x7fa940066460 secure :-1 s=THROTTLE_DONE pgs=305 cs=0 l=1 rx=0x7fa93c000b80 tx=0x7fa93c010860).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:19:17.514 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.504+0000 7fa9337fe700 1 -- 172.21.15.170:0/162160517 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 4) v1 ==== 29603+0+0 (secure 0 0 0) 0x7fa93c00a320 con 0x7fa94008d3e0 2020-03-19T21:19:17.514 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.504+0000 7fa9337fe700 1 --2- 172.21.15.170:0/162160517 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7fa9200287c0 0x7fa92002ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:19:17.514 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.504+0000 7fa9337fe700 1 -- 172.21.15.170:0/162160517 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 3786+0+0 (secure 0 0 0) 0x7fa93c0340b0 con 0x7fa94008d3e0 2020-03-19T21:19:17.514 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.504+0000 7fa94645a700 1 --2- 172.21.15.170:0/162160517 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7fa9200287c0 0x7fa92002ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:19:17.514 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.504+0000 7fa94645a700 1 --2- 172.21.15.170:0/162160517 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7fa9200287c0 0x7fa92002ac80 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7fa934000b80 tx=0x7fa934007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:19:17.514 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.504+0000 7fa94645a700 1 --2- 172.21.15.170:0/162160517 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7fa9200287c0 0x7fa92002ac80 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7fa934000b80 tx=0x7fa934007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:19:17.515 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.505+0000 7fa94645a700 1 --2- 172.21.15.170:0/162160517 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7fa9200287c0 0x7fa92002ac80 secure :-1 s=READY pgs=75 cs=0 l=1 rx=0x7fa934000b80 tx=0x7fa934007a40).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:19:17.515 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.505+0000 7fa9486be700 1 -- 172.21.15.170:0/162160517 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa940056db0 con 0x7fa94008d3e0 2020-03-19T21:19:17.515 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.508+0000 7fa945c59700 1 --2- 172.21.15.170:0/162160517 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa94008d3e0 0x7fa940066460 secure :-1 s=THROTTLE_DONE pgs=305 cs=0 l=1 rx=0x7fa93c000b80 tx=0x7fa93c010860).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:19:17.515 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.508+0000 7fa9337fe700 1 -- 172.21.15.170:0/162160517 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+126772 (secure 0 0 0) 0x7fa93c006220 con 0x7fa94008d3e0 2020-03-19T21:19:17.644 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.643+0000 7fa9486be700 1 -- 172.21.15.170:0/162160517 --> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa94011f080 con 0x7fa9200287c0 2020-03-19T21:19:17.649 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.645+0000 7fa94645a700 1 --2- 172.21.15.170:0/162160517 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7fa9200287c0 0x7fa92002ac80 secure :-1 s=THROTTLE_DONE pgs=75 cs=0 l=1 rx=0x7fa934000b80 tx=0x7fa934007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:19:17.649 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.645+0000 7fa9337fe700 1 -- 172.21.15.170:0/162160517 <== mgr.4102 v2:172.21.15.170:6824/20154 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+76880 (secure 0 0 0) 0x7fa934008660 con 0x7fa9200287c0 2020-03-19T21:19:17.650 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.647+0000 7fa9486be700 1 -- 172.21.15.170:0/162160517 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7fa9200287c0 msgr2=0x7fa92002ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:19:17.650 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.647+0000 7fa9486be700 1 --2- 172.21.15.170:0/162160517 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7fa9200287c0 0x7fa92002ac80 secure :-1 s=READY pgs=75 cs=0 l=1 rx=0x7fa934000b80 tx=0x7fa934007a40).stop 2020-03-19T21:19:17.650 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.647+0000 7fa9486be700 1 -- 172.21.15.170:0/162160517 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa94008d3e0 msgr2=0x7fa940066460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:19:17.651 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.647+0000 7fa9486be700 1 --2- 172.21.15.170:0/162160517 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa94008d3e0 0x7fa940066460 secure :-1 s=READY pgs=305 cs=0 l=1 rx=0x7fa93c000b80 tx=0x7fa93c010860).stop 2020-03-19T21:19:17.651 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.647+0000 7fa9486be700 1 -- 172.21.15.170:0/162160517 shutdown_connections 2020-03-19T21:19:17.651 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.647+0000 7fa9486be700 1 --2- 172.21.15.170:0/162160517 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7fa9200287c0 0x7fa92002ac80 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:19:17.652 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.647+0000 7fa9486be700 1 --2- 172.21.15.170:0/162160517 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa94008d3e0 0x7fa940066460 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:19:17.652 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.647+0000 7fa9486be700 1 -- 172.21.15.170:0/162160517 >> 172.21.15.170:0/162160517 conn(0x7fa940091540 msgr2=0x7fa9400935b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:19:17.652 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.647+0000 7fa9486be700 1 -- 172.21.15.170:0/162160517 shutdown_connections 2020-03-19T21:19:17.652 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.647+0000 7fa9486be700 1 -- 172.21.15.170:0/162160517 wait complete. 2020-03-19T21:19:17.653 INFO:teuthology.orchestra.run.smithi170.stderr:dumped all 2020-03-19T21:19:17.653 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:19:17.659 INFO:teuthology.orchestra.run.smithi170.stdout:{"pg_ready":true,"pg_map":{"version":58,"stamp":"2020-03-19T21:19:16.521315+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2718,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":4620,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":57,"ondisk_log_size":57,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":314572800,"kb_used":3146416,"kb_used_data":664,"kb_used_omap":0,"kb_used_meta":3145728,"kb_avail":311426384,"statfs":{"total":322122547200,"available":318900617216,"internally_reserved":3221225472,"allocated":679936,"data_stored":238689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":3221225472},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002467"},"pg_stats":[{"pgid":"4.0","version":"0'0","reported_seq":"16","reported_epoch":"24","state":"active+clean","last_fresh":"2020-03-19T21:18:47.372788+0000","last_change":"2020-03-19T21:18:47.372844+0000","last_active":"2020-03-19T21:18:47.372788+0000","last_peered":"2020-03-19T21:18:47.372788+0000","last_clean":"2020-03-19T21:18:47.372788+0000","last_became_active":"2020-03-19T21:18:24.224136+0000","last_became_peered":"2020-03-19T21:18:24.224136+0000","last_unstale":"2020-03-19T21:18:47.372788+0000","last_undegraded":"2020-03-19T21:18:47.372788+0000","last_fullsized":"2020-03-19T21:18:47.372788+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:47.372783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:47.372783+0000","last_clean_scrub_stamp":"2020-03-19T21:18:47.372783+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":"19","reported_epoch":"24","state":"active+clean","last_fresh":"2020-03-19T21:18:48.336985+0000","last_change":"2020-03-19T21:18:48.337038+0000","last_active":"2020-03-19T21:18:48.336985+0000","last_peered":"2020-03-19T21:18:48.336985+0000","last_clean":"2020-03-19T21:18:48.336985+0000","last_became_active":"2020-03-19T21:18:13.525041+0000","last_became_peered":"2020-03-19T21:18:13.525041+0000","last_unstale":"2020-03-19T21:18:48.336985+0000","last_undegraded":"2020-03-19T21:18:48.336985+0000","last_fullsized":"2020-03-19T21:18:48.336985+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:48.336979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:48.336979+0000","last_clean_scrub_stamp":"2020-03-19T21:18:48.336979+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":"23","reported_epoch":"24","state":"active+clean","last_fresh":"2020-03-19T21:18:41.280273+0000","last_change":"2020-03-19T21:18:41.280344+0000","last_active":"2020-03-19T21:18:41.280273+0000","last_peered":"2020-03-19T21:18:41.280273+0000","last_clean":"2020-03-19T21:18:41.280273+0000","last_became_active":"2020-03-19T21:18:10.186584+0000","last_became_peered":"2020-03-19T21:18:10.186584+0000","last_unstale":"2020-03-19T21:18:41.280273+0000","last_undegraded":"2020-03-19T21:18:41.280273+0000","last_fullsized":"2020-03-19T21:18:41.280273+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:41.280268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:41.280268+0000","last_clean_scrub_stamp":"2020-03-19T21:18:41.280268+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"24'6","reported_seq":"27","reported_epoch":"24","state":"active+clean","last_fresh":"2020-03-19T21:18:55.565763+0000","last_change":"2020-03-19T21:18:55.565821+0000","last_active":"2020-03-19T21:18:55.565763+0000","last_peered":"2020-03-19T21:18:55.565763+0000","last_clean":"2020-03-19T21:18:55.565763+0000","last_became_active":"2020-03-19T21:18:13.197671+0000","last_became_peered":"2020-03-19T21:18:13.197671+0000","last_unstale":"2020-03-19T21:18:55.565763+0000","last_undegraded":"2020-03-19T21:18:55.565763+0000","last_fullsized":"2020-03-19T21:18:55.565763+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"24'6","last_scrub_stamp":"2020-03-19T21:18:55.565757+0000","last_deep_scrub":"24'6","last_deep_scrub_stamp":"2020-03-19T21:18:55.565757+0000","last_clean_scrub_stamp":"2020-03-19T21:18:55.565757+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":"16","reported_epoch":"24","state":"active+clean","last_fresh":"2020-03-19T21:18:49.352476+0000","last_change":"2020-03-19T21:18:49.352526+0000","last_active":"2020-03-19T21:18:49.352476+0000","last_peered":"2020-03-19T21:18:49.352476+0000","last_clean":"2020-03-19T21:18:49.352476+0000","last_became_active":"2020-03-19T21:18:24.224025+0000","last_became_peered":"2020-03-19T21:18:24.224025+0000","last_unstale":"2020-03-19T21:18:49.352476+0000","last_undegraded":"2020-03-19T21:18:49.352476+0000","last_fullsized":"2020-03-19T21:18:49.352476+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:49.352468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:49.352468+0000","last_clean_scrub_stamp":"2020-03-19T21:18:49.352468+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":"19","reported_epoch":"24","state":"active+clean","last_fresh":"2020-03-19T21:18:45.305647+0000","last_change":"2020-03-19T21:18:45.305700+0000","last_active":"2020-03-19T21:18:45.305647+0000","last_peered":"2020-03-19T21:18:45.305647+0000","last_clean":"2020-03-19T21:18:45.305647+0000","last_became_active":"2020-03-19T21:18:13.524563+0000","last_became_peered":"2020-03-19T21:18:13.524563+0000","last_unstale":"2020-03-19T21:18:45.305647+0000","last_undegraded":"2020-03-19T21:18:45.305647+0000","last_fullsized":"2020-03-19T21:18:45.305647+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:45.305641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:45.305641+0000","last_clean_scrub_stamp":"2020-03-19T21:18:45.305641+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":"23","reported_epoch":"24","state":"active+clean","last_fresh":"2020-03-19T21:18:41.401276+0000","last_change":"2020-03-19T21:18:41.401331+0000","last_active":"2020-03-19T21:18:41.401276+0000","last_peered":"2020-03-19T21:18:41.401276+0000","last_clean":"2020-03-19T21:18:41.401276+0000","last_became_active":"2020-03-19T21:18:10.183810+0000","last_became_peered":"2020-03-19T21:18:10.183810+0000","last_unstale":"2020-03-19T21:18:41.401276+0000","last_undegraded":"2020-03-19T21:18:41.401276+0000","last_fullsized":"2020-03-19T21:18:41.401276+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:41.401271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:41.401271+0000","last_clean_scrub_stamp":"2020-03-19T21:18:41.401271+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"24'11","reported_seq":"32","reported_epoch":"24","state":"active+clean","last_fresh":"2020-03-19T21:19:00.492648+0000","last_change":"2020-03-19T21:19:00.492703+0000","last_active":"2020-03-19T21:19:00.492648+0000","last_peered":"2020-03-19T21:19:00.492648+0000","last_clean":"2020-03-19T21:19:00.492648+0000","last_became_active":"2020-03-19T21:18:13.195896+0000","last_became_peered":"2020-03-19T21:18:13.195896+0000","last_unstale":"2020-03-19T21:19:00.492648+0000","last_undegraded":"2020-03-19T21:19:00.492648+0000","last_fullsized":"2020-03-19T21:19:00.492648+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"24'11","last_scrub_stamp":"2020-03-19T21:19:00.492641+0000","last_deep_scrub":"24'11","last_deep_scrub_stamp":"2020-03-19T21:19:00.492641+0000","last_clean_scrub_stamp":"2020-03-19T21:19:00.492641+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":566,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":"19","reported_epoch":"24","state":"active+clean","last_fresh":"2020-03-19T21:18:41.293088+0000","last_change":"2020-03-19T21:18:41.293146+0000","last_active":"2020-03-19T21:18:41.293088+0000","last_peered":"2020-03-19T21:18:41.293088+0000","last_clean":"2020-03-19T21:18:41.293088+0000","last_became_active":"2020-03-19T21:18:13.523806+0000","last_became_peered":"2020-03-19T21:18:13.523806+0000","last_unstale":"2020-03-19T21:18:41.293088+0000","last_undegraded":"2020-03-19T21:18:41.293088+0000","last_fullsized":"2020-03-19T21:18:41.293088+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:41.293083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:41.293083+0000","last_clean_scrub_stamp":"2020-03-19T21:18:41.293083+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":"23","reported_epoch":"24","state":"active+clean","last_fresh":"2020-03-19T21:18:39.363113+0000","last_change":"2020-03-19T21:18:39.363190+0000","last_active":"2020-03-19T21:18:39.363113+0000","last_peered":"2020-03-19T21:18:39.363113+0000","last_clean":"2020-03-19T21:18:39.363113+0000","last_became_active":"2020-03-19T21:18:10.183788+0000","last_became_peered":"2020-03-19T21:18:10.183788+0000","last_unstale":"2020-03-19T21:18:39.363113+0000","last_undegraded":"2020-03-19T21:18:39.363113+0000","last_fullsized":"2020-03-19T21:18:39.363113+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:39.363109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:39.363109+0000","last_clean_scrub_stamp":"2020-03-19T21:18:39.363109+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"16'8","reported_seq":"28","reported_epoch":"24","state":"active+clean","last_fresh":"2020-03-19T21:18:42.318384+0000","last_change":"2020-03-19T21:18:42.318461+0000","last_active":"2020-03-19T21:18:42.318384+0000","last_peered":"2020-03-19T21:18:42.318384+0000","last_clean":"2020-03-19T21:18:42.318384+0000","last_became_active":"2020-03-19T21:18:13.196823+0000","last_became_peered":"2020-03-19T21:18:13.196823+0000","last_unstale":"2020-03-19T21:18:42.318384+0000","last_undegraded":"2020-03-19T21:18:42.318384+0000","last_fullsized":"2020-03-19T21:18:42.318384+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"16'8","last_scrub_stamp":"2020-03-19T21:18:42.318376+0000","last_deep_scrub":"16'8","last_deep_scrub_stamp":"2020-03-19T21:18:42.318376+0000","last_clean_scrub_stamp":"2020-03-19T21:18:42.318376+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":"20","reported_epoch":"24","state":"active+clean","last_fresh":"2020-03-19T21:19:01.457686+0000","last_change":"2020-03-19T21:19:01.457741+0000","last_active":"2020-03-19T21:19:01.457686+0000","last_peered":"2020-03-19T21:19:01.457686+0000","last_clean":"2020-03-19T21:19:01.457686+0000","last_became_active":"2020-03-19T21:18:13.527916+0000","last_became_peered":"2020-03-19T21:18:13.527916+0000","last_unstale":"2020-03-19T21:19:01.457686+0000","last_undegraded":"2020-03-19T21:19:01.457686+0000","last_fullsized":"2020-03-19T21:19:01.457686+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:19:01.457683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:19:01.457683+0000","last_clean_scrub_stamp":"2020-03-19T21:19:01.457683+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":"22","reported_epoch":"24","state":"active+clean","last_fresh":"2020-03-19T21:18:38.275259+0000","last_change":"2020-03-19T21:18:38.275323+0000","last_active":"2020-03-19T21:18:38.275259+0000","last_peered":"2020-03-19T21:18:38.275259+0000","last_clean":"2020-03-19T21:18:38.275259+0000","last_became_active":"2020-03-19T21:18:10.187570+0000","last_became_peered":"2020-03-19T21:18:10.187570+0000","last_unstale":"2020-03-19T21:18:38.275259+0000","last_undegraded":"2020-03-19T21:18:38.275259+0000","last_fullsized":"2020-03-19T21:18:38.275259+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:38.275254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:38.275254+0000","last_clean_scrub_stamp":"2020-03-19T21:18:38.275254+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"20","reported_epoch":"24","state":"active+clean","last_fresh":"2020-03-19T21:18:43.343893+0000","last_change":"2020-03-19T21:18:43.343958+0000","last_active":"2020-03-19T21:18:43.343893+0000","last_peered":"2020-03-19T21:18:43.343893+0000","last_clean":"2020-03-19T21:18:43.343893+0000","last_became_active":"2020-03-19T21:18:13.197224+0000","last_became_peered":"2020-03-19T21:18:13.197224+0000","last_unstale":"2020-03-19T21:18:43.343893+0000","last_undegraded":"2020-03-19T21:18:43.343893+0000","last_fullsized":"2020-03-19T21:18:43.343893+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:43.343884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:43.343884+0000","last_clean_scrub_stamp":"2020-03-19T21:18:43.343884+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":"20","reported_epoch":"24","state":"active+clean","last_fresh":"2020-03-19T21:19:03.434759+0000","last_change":"2020-03-19T21:19:03.434822+0000","last_active":"2020-03-19T21:19:03.434759+0000","last_peered":"2020-03-19T21:19:03.434759+0000","last_clean":"2020-03-19T21:19:03.434759+0000","last_became_active":"2020-03-19T21:18:13.523902+0000","last_became_peered":"2020-03-19T21:18:13.523902+0000","last_unstale":"2020-03-19T21:19:03.434759+0000","last_undegraded":"2020-03-19T21:19:03.434759+0000","last_fullsized":"2020-03-19T21:19:03.434759+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:19:03.434752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:19:03.434752+0000","last_clean_scrub_stamp":"2020-03-19T21:19:03.434752+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":"23","reported_epoch":"24","state":"active+clean","last_fresh":"2020-03-19T21:18:38.360368+0000","last_change":"2020-03-19T21:18:38.360418+0000","last_active":"2020-03-19T21:18:38.360368+0000","last_peered":"2020-03-19T21:18:38.360368+0000","last_clean":"2020-03-19T21:18:38.360368+0000","last_became_active":"2020-03-19T21:18:10.184433+0000","last_became_peered":"2020-03-19T21:18:10.184433+0000","last_unstale":"2020-03-19T21:18:38.360368+0000","last_undegraded":"2020-03-19T21:18:38.360368+0000","last_fullsized":"2020-03-19T21:18:38.360368+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:38.360362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:38.360362+0000","last_clean_scrub_stamp":"2020-03-19T21:18:38.360362+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"24'8","reported_seq":"29","reported_epoch":"24","state":"active+clean","last_fresh":"2020-03-19T21:18:47.489117+0000","last_change":"2020-03-19T21:18:47.489175+0000","last_active":"2020-03-19T21:18:47.489117+0000","last_peered":"2020-03-19T21:18:47.489117+0000","last_clean":"2020-03-19T21:18:47.489117+0000","last_became_active":"2020-03-19T21:18:13.194849+0000","last_became_peered":"2020-03-19T21:18:13.194849+0000","last_unstale":"2020-03-19T21:18:47.489117+0000","last_undegraded":"2020-03-19T21:18:47.489117+0000","last_fullsized":"2020-03-19T21:18:47.489117+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"24'8","last_scrub_stamp":"2020-03-19T21:18:47.489109+0000","last_deep_scrub":"24'8","last_deep_scrub_stamp":"2020-03-19T21:18:47.489109+0000","last_clean_scrub_stamp":"2020-03-19T21:18:47.489109+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":4620,"num_omap_keys":10,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":"19","reported_epoch":"24","state":"active+clean","last_fresh":"2020-03-19T21:18:42.275966+0000","last_change":"2020-03-19T21:18:42.276022+0000","last_active":"2020-03-19T21:18:42.275966+0000","last_peered":"2020-03-19T21:18:42.275966+0000","last_clean":"2020-03-19T21:18:42.275966+0000","last_became_active":"2020-03-19T21:18:13.522937+0000","last_became_peered":"2020-03-19T21:18:13.522937+0000","last_unstale":"2020-03-19T21:18:42.275966+0000","last_undegraded":"2020-03-19T21:18:42.275966+0000","last_fullsized":"2020-03-19T21:18:42.275966+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:42.275962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:42.275962+0000","last_clean_scrub_stamp":"2020-03-19T21:18:42.275962+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":"23","reported_epoch":"24","state":"active+clean","last_fresh":"2020-03-19T21:18:39.239015+0000","last_change":"2020-03-19T21:18:39.239071+0000","last_active":"2020-03-19T21:18:39.239015+0000","last_peered":"2020-03-19T21:18:39.239015+0000","last_clean":"2020-03-19T21:18:39.239015+0000","last_became_active":"2020-03-19T21:18:10.183404+0000","last_became_peered":"2020-03-19T21:18:10.183404+0000","last_unstale":"2020-03-19T21:18:39.239015+0000","last_undegraded":"2020-03-19T21:18:39.239015+0000","last_fullsized":"2020-03-19T21:18:39.239015+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:39.239009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:39.239009+0000","last_clean_scrub_stamp":"2020-03-19T21:18:39.239009+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"16'1","reported_seq":"21","reported_epoch":"24","state":"active+clean","last_fresh":"2020-03-19T21:18:40.270522+0000","last_change":"2020-03-19T21:18:40.270588+0000","last_active":"2020-03-19T21:18:40.270522+0000","last_peered":"2020-03-19T21:18:40.270522+0000","last_clean":"2020-03-19T21:18:40.270522+0000","last_became_active":"2020-03-19T21:18:13.198498+0000","last_became_peered":"2020-03-19T21:18:13.198498+0000","last_unstale":"2020-03-19T21:18:40.270522+0000","last_undegraded":"2020-03-19T21:18:40.270522+0000","last_fullsized":"2020-03-19T21:18:40.270522+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"16'1","last_scrub_stamp":"2020-03-19T21:18:40.270516+0000","last_deep_scrub":"16'1","last_deep_scrub_stamp":"2020-03-19T21:18:40.270516+0000","last_clean_scrub_stamp":"2020-03-19T21:18:40.270516+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":"16","reported_epoch":"24","state":"active+clean","last_fresh":"2020-03-19T21:18:49.332432+0000","last_change":"2020-03-19T21:18:49.332513+0000","last_active":"2020-03-19T21:18:49.332432+0000","last_peered":"2020-03-19T21:18:49.332432+0000","last_clean":"2020-03-19T21:18:49.332432+0000","last_became_active":"2020-03-19T21:18:24.224328+0000","last_became_peered":"2020-03-19T21:18:24.224328+0000","last_unstale":"2020-03-19T21:18:49.332432+0000","last_undegraded":"2020-03-19T21:18:49.332432+0000","last_fullsized":"2020-03-19T21:18:49.332432+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:49.332427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:49.332427+0000","last_clean_scrub_stamp":"2020-03-19T21:18:49.332427+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":"20","reported_epoch":"24","state":"active+clean","last_fresh":"2020-03-19T21:19:05.462809+0000","last_change":"2020-03-19T21:19:05.462868+0000","last_active":"2020-03-19T21:19:05.462809+0000","last_peered":"2020-03-19T21:19:05.462809+0000","last_clean":"2020-03-19T21:19:05.462809+0000","last_became_active":"2020-03-19T21:18:13.524437+0000","last_became_peered":"2020-03-19T21:18:13.524437+0000","last_unstale":"2020-03-19T21:19:05.462809+0000","last_undegraded":"2020-03-19T21:19:05.462809+0000","last_fullsized":"2020-03-19T21:19:05.462809+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:19:05.462806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:19:05.462806+0000","last_clean_scrub_stamp":"2020-03-19T21:19:05.462806+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":"23","reported_epoch":"24","state":"active+clean","last_fresh":"2020-03-19T21:18:43.440965+0000","last_change":"2020-03-19T21:18:43.441016+0000","last_active":"2020-03-19T21:18:43.440965+0000","last_peered":"2020-03-19T21:18:43.440965+0000","last_clean":"2020-03-19T21:18:43.440965+0000","last_became_active":"2020-03-19T21:18:10.184549+0000","last_became_peered":"2020-03-19T21:18:10.184549+0000","last_unstale":"2020-03-19T21:18:43.440965+0000","last_undegraded":"2020-03-19T21:18:43.440965+0000","last_fullsized":"2020-03-19T21:18:43.440965+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:43.440960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:43.440960+0000","last_clean_scrub_stamp":"2020-03-19T21:18:43.440960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"24'10","reported_seq":"31","reported_epoch":"24","state":"active+clean","last_fresh":"2020-03-19T21:18:52.575209+0000","last_change":"2020-03-19T21:18:52.575264+0000","last_active":"2020-03-19T21:18:52.575209+0000","last_peered":"2020-03-19T21:18:52.575209+0000","last_clean":"2020-03-19T21:18:52.575209+0000","last_became_active":"2020-03-19T21:18:13.195947+0000","last_became_peered":"2020-03-19T21:18:13.195947+0000","last_unstale":"2020-03-19T21:18:52.575209+0000","last_undegraded":"2020-03-19T21:18:52.575209+0000","last_fullsized":"2020-03-19T21:18:52.575209+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"24'10","last_scrub_stamp":"2020-03-19T21:18:52.575203+0000","last_deep_scrub":"24'10","last_deep_scrub_stamp":"2020-03-19T21:18:52.575203+0000","last_clean_scrub_stamp":"2020-03-19T21:18:52.575203+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":160,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":"17","reported_epoch":"24","state":"active+clean","last_fresh":"2020-03-19T21:19:07.468577+0000","last_change":"2020-03-19T21:19:07.468636+0000","last_active":"2020-03-19T21:19:07.468577+0000","last_peered":"2020-03-19T21:19:07.468577+0000","last_clean":"2020-03-19T21:19:07.468577+0000","last_became_active":"2020-03-19T21:18:24.223594+0000","last_became_peered":"2020-03-19T21:18:24.223594+0000","last_unstale":"2020-03-19T21:19:07.468577+0000","last_undegraded":"2020-03-19T21:19:07.468577+0000","last_fullsized":"2020-03-19T21:19:07.468577+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:19:07.468572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:19:07.468572+0000","last_clean_scrub_stamp":"2020-03-19T21:19:07.468572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":"19","reported_epoch":"24","state":"active+clean","last_fresh":"2020-03-19T21:18:45.371339+0000","last_change":"2020-03-19T21:18:45.371397+0000","last_active":"2020-03-19T21:18:45.371339+0000","last_peered":"2020-03-19T21:18:45.371339+0000","last_clean":"2020-03-19T21:18:45.371339+0000","last_became_active":"2020-03-19T21:18:13.524938+0000","last_became_peered":"2020-03-19T21:18:13.524938+0000","last_unstale":"2020-03-19T21:18:45.371339+0000","last_undegraded":"2020-03-19T21:18:45.371339+0000","last_fullsized":"2020-03-19T21:18:45.371339+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:45.371335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:45.371335+0000","last_clean_scrub_stamp":"2020-03-19T21:18:45.371335+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":"23","reported_epoch":"24","state":"active+clean","last_fresh":"2020-03-19T21:18:45.446348+0000","last_change":"2020-03-19T21:18:45.446415+0000","last_active":"2020-03-19T21:18:45.446348+0000","last_peered":"2020-03-19T21:18:45.446348+0000","last_clean":"2020-03-19T21:18:45.446348+0000","last_became_active":"2020-03-19T21:18:10.181738+0000","last_became_peered":"2020-03-19T21:18:10.181738+0000","last_unstale":"2020-03-19T21:18:45.446348+0000","last_undegraded":"2020-03-19T21:18:45.446348+0000","last_fullsized":"2020-03-19T21:18:45.446348+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T21:18:45.446342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T21:18:45.446342+0000","last_clean_scrub_stamp":"2020-03-19T21:18:45.446342+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"24'13","reported_seq":"34","reported_epoch":"24","state":"active+clean","last_fresh":"2020-03-19T21:18:49.510676+0000","last_change":"2020-03-19T21:18:49.510739+0000","last_active":"2020-03-19T21:18:49.510676+0000","last_peered":"2020-03-19T21:18:49.510676+0000","last_clean":"2020-03-19T21:18:49.510676+0000","last_became_active":"2020-03-19T21:18:13.196089+0000","last_became_peered":"2020-03-19T21:18:13.196089+0000","last_unstale":"2020-03-19T21:18:49.510676+0000","last_undegraded":"2020-03-19T21:18:49.510676+0000","last_fullsized":"2020-03-19T21:18:49.510676+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"24'13","last_scrub_stamp":"2020-03-19T21:18:49.510667+0000","last_deep_scrub":"24'13","last_deep_scrub_stamp":"2020-03-19T21:18:49.510667+0000","last_clean_scrub_stamp":"2020-03-19T21:18:49.510667+0000","log_size":13,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1958,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":4,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":8,"acting":8,"num_store_stats":3},{"poolid":3,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":2718,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":4620,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":5436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":57,"ondisk_log_size":57,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738386,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":1048788,"kb_used_data":204,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":103808812,"statfs":{"total":107374182400,"available":106300223488,"internally_reserved":1073741824,"allocated":208896,"data_stored":77785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 19 21:19:08 2020","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.54100000000000004}]},{"osd":1,"last update":"Thu Mar 19 21:19:08 2020","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.879}]}]},{"osd":0,"up_from":8,"seq":34359738385,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":1048812,"kb_used_data":228,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":103808788,"statfs":{"total":107374182400,"available":106300198912,"internally_reserved":1073741824,"allocated":233472,"data_stored":80435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Mar 19 21:19:09 2020","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":2.8809999999999998,"5min":2.8809999999999998,"15min":2.8809999999999998},"last":0.629}]},{"osd":2,"last update":"Thu Mar 19 21:19:09 2020","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":3.8149999999999999,"5min":3.8149999999999999,"15min":3.8149999999999999},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.5649999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.69399999999999995}]}]},{"osd":1,"up_from":8,"seq":34359738385,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":1048816,"kb_used_data":232,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":103808784,"statfs":{"total":107374182400,"available":106300194816,"internally_reserved":1073741824,"allocated":237568,"data_stored":80469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 19 21:19:05 2020","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.65500000000000003}]},{"osd":2,"last update":"Thu Mar 19 21:19:05 2020","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.78300000000000003}]}]}]}} 2020-03-19T21:19:17.666 INFO:teuthology.orchestra.run.smithi170:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2020-03-19T21:19:17.776 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.770+0000 7f43427d1700 1 Processor -- start 2020-03-19T21:19:17.777 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.770+0000 7f43427d1700 1 -- start start 2020-03-19T21:19:17.777 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.770+0000 7f43427d1700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f433c08f010 0x7f433c08f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T21:19:17.777 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.770+0000 7f43427d1700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f433c0902c0 con 0x7f433c08f850 2020-03-19T21:19:17.777 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.770+0000 7f43427d1700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f433c0904f0 con 0x7f433c08f010 2020-03-19T21:19:17.777 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.770+0000 7f4340d6e700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f433c08f010 0x7f433c08f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:19:17.777 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.770+0000 7f4340d6e700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f433c08f010 0x7f433c08f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53408/0 (socket says 172.21.15.170:53408) 2020-03-19T21:19:17.778 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.770+0000 7f433bfff700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f433c08f850 0x7f433c136ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:54172/0 (socket says 172.21.15.170:54172) 2020-03-19T21:19:17.778 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.770+0000 7f4340d6e700 1 -- 172.21.15.170:0/3824753367 learned_addr learned my addr 172.21.15.170:0/3824753367 (peer_addr_for_me v2:172.21.15.170:0/0) 2020-03-19T21:19:17.778 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.771+0000 7f4340d6e700 1 -- 172.21.15.170:0/3824753367 >> v1:172.21.15.170:6789/0 conn(0x7f433c08f850 legacy=0x7f433c136ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:19:17.778 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.771+0000 7f4340d6e700 1 -- 172.21.15.170:0/3824753367 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f433c0954e0 con 0x7f433c08f010 2020-03-19T21:19:17.778 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.771+0000 7f433b7fe700 1 -- 172.21.15.170:0/3824753367 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2478801634 0 0) 0x7f433c0902c0 con 0x7f433c08f850 2020-03-19T21:19:17.778 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.771+0000 7f4340d6e700 1 --2- 172.21.15.170:0/3824753367 >> v2:172.21.15.170:3300/0 conn(0x7f433c08f010 0x7f433c08f3f0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f432c002f30 tx=0x7f432c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:19:17.778 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.771+0000 7f4340d6e700 1 --2- 172.21.15.170:0/3824753367 >> v2:172.21.15.170:3300/0 conn(0x7f433c08f010 0x7f433c08f3f0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f432c002f30 tx=0x7f432c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:19:17.779 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.771+0000 7f4340d6e700 1 --2- 172.21.15.170:0/3824753367 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f433c08f010 0x7f433c08f3f0 secure :-1 s=READY pgs=307 cs=0 l=1 rx=0x7f432c002f30 tx=0x7f432c007a40).ready entity=mon.0 client_cookie=e6a39e50655271d6 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:19:17.779 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.771+0000 7f4340d6e700 1 --2- 172.21.15.170:0/3824753367 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f433c08f010 0x7f433c08f3f0 secure :-1 s=THROTTLE_DONE pgs=307 cs=0 l=1 rx=0x7f432c002f30 tx=0x7f432c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:19:17.779 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.771+0000 7f433b7fe700 1 -- 172.21.15.170:0/3824753367 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f432c0114b0 con 0x7f433c08f010 2020-03-19T21:19:17.779 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.771+0000 7f4340d6e700 1 --2- 172.21.15.170:0/3824753367 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f433c08f010 0x7f433c08f3f0 secure :-1 s=THROTTLE_DONE pgs=307 cs=0 l=1 rx=0x7f432c002f30 tx=0x7f432c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:19:17.779 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.771+0000 7f433b7fe700 1 -- 172.21.15.170:0/3824753367 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f432c011610 con 0x7f433c08f010 2020-03-19T21:19:17.779 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.771+0000 7f4340d6e700 1 --2- 172.21.15.170:0/3824753367 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f433c08f010 0x7f433c08f3f0 secure :-1 s=THROTTLE_DONE pgs=307 cs=0 l=1 rx=0x7f432c002f30 tx=0x7f432c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:19:17.779 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.771+0000 7f433b7fe700 1 -- 172.21.15.170:0/3824753367 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f432c011930 con 0x7f433c08f010 2020-03-19T21:19:17.780 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.771+0000 7f43427d1700 1 -- 172.21.15.170:0/3824753367 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f433c08f010 msgr2=0x7f433c08f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:19:17.780 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.771+0000 7f43427d1700 1 --2- 172.21.15.170:0/3824753367 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f433c08f010 0x7f433c08f3f0 secure :-1 s=READY pgs=307 cs=0 l=1 rx=0x7f432c002f30 tx=0x7f432c007a40).stop 2020-03-19T21:19:17.780 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.771+0000 7f43427d1700 1 -- 172.21.15.170:0/3824753367 shutdown_connections 2020-03-19T21:19:17.780 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.771+0000 7f43427d1700 1 --2- 172.21.15.170:0/3824753367 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f433c08f010 0x7f433c08f3f0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:19:17.780 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.771+0000 7f43427d1700 1 -- 172.21.15.170:0/3824753367 >> 172.21.15.170:0/3824753367 conn(0x7f433c095090 msgr2=0x7f433c091110 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:19:17.780 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.771+0000 7f43427d1700 1 -- 172.21.15.170:0/3824753367 shutdown_connections 2020-03-19T21:19:17.780 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.771+0000 7f43427d1700 1 -- 172.21.15.170:0/3824753367 wait complete. 2020-03-19T21:19:17.781 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.772+0000 7f43427d1700 1 Processor -- start 2020-03-19T21:19:17.781 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.772+0000 7f43427d1700 1 -- start start 2020-03-19T21:19:17.781 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.773+0000 7f43427d1700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f433c08f010 0x7f433c0707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:19:17.781 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.773+0000 7f43427d1700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f433c071e40 con 0x7f433c08f850 2020-03-19T21:19:17.781 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.773+0000 7f43427d1700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f433c077cc0 con 0x7f433c08f010 2020-03-19T21:19:17.781 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.773+0000 7f4340d6e700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f433c08f010 0x7f433c0707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:19:17.781 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.773+0000 7f433bfff700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f433c08f850 0x7f433c070c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:54178/0 (socket says 172.21.15.170:54178) 2020-03-19T21:19:17.782 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.773+0000 7f4340d6e700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f433c08f010 0x7f433c0707a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53412/0 (socket says 172.21.15.170:53412) 2020-03-19T21:19:17.782 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.773+0000 7f433bfff700 1 -- 172.21.15.170:0/1931309176 learned_addr learned my addr 172.21.15.170:0/1931309176 (peer_addr_for_me v1:172.21.15.170:0/0) 2020-03-19T21:19:17.782 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.773+0000 7f4340d6e700 1 -- 172.21.15.170:0/1931309176 >> v1:172.21.15.170:6789/0 conn(0x7f433c08f850 legacy=0x7f433c070c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:19:17.782 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.773+0000 7f4340d6e700 1 -- 172.21.15.170:0/1931309176 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f433c0937d0 con 0x7f433c08f010 2020-03-19T21:19:17.782 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.773+0000 7f43397fa700 1 -- 172.21.15.170:0/1931309176 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3194081481 0 0) 0x7f433c071e40 con 0x7f433c08f850 2020-03-19T21:19:17.782 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.773+0000 7f4340d6e700 1 --2- 172.21.15.170:0/1931309176 >> v2:172.21.15.170:3300/0 conn(0x7f433c08f010 0x7f433c0707a0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f432c002f30 tx=0x7f432c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:19:17.782 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.773+0000 7f4340d6e700 1 --2- 172.21.15.170:0/1931309176 >> v2:172.21.15.170:3300/0 conn(0x7f433c08f010 0x7f433c0707a0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f432c002f30 tx=0x7f432c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:19:17.783 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.773+0000 7f4340d6e700 1 --2- 172.21.15.170:0/1931309176 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f433c08f010 0x7f433c0707a0 secure :-1 s=READY pgs=309 cs=0 l=1 rx=0x7f432c002f30 tx=0x7f432c007a40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:19:17.783 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.773+0000 7f4340d6e700 1 --2- 172.21.15.170:0/1931309176 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f433c08f010 0x7f433c0707a0 secure :-1 s=THROTTLE_DONE pgs=309 cs=0 l=1 rx=0x7f432c002f30 tx=0x7f432c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:19:17.783 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.773+0000 7f43397fa700 1 -- 172.21.15.170:0/1931309176 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f432c00f870 con 0x7f433c08f010 2020-03-19T21:19:17.783 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.773+0000 7f4340d6e700 1 --2- 172.21.15.170:0/1931309176 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f433c08f010 0x7f433c0707a0 secure :-1 s=THROTTLE_DONE pgs=309 cs=0 l=1 rx=0x7f432c002f30 tx=0x7f432c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:19:17.783 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.773+0000 7f43397fa700 1 -- 172.21.15.170:0/1931309176 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f432c007150 con 0x7f433c08f010 2020-03-19T21:19:17.783 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.773+0000 7f4340d6e700 1 --2- 172.21.15.170:0/1931309176 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f433c08f010 0x7f433c0707a0 secure :-1 s=THROTTLE_DONE pgs=309 cs=0 l=1 rx=0x7f432c002f30 tx=0x7f432c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:19:17.783 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.773+0000 7f43427d1700 1 -- 172.21.15.170:0/1931309176 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f433c076ec0 con 0x7f433c08f010 2020-03-19T21:19:17.784 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.773+0000 7f43397fa700 1 -- 172.21.15.170:0/1931309176 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f432c0073f0 con 0x7f433c08f010 2020-03-19T21:19:17.784 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.774+0000 7f43427d1700 1 -- 172.21.15.170:0/1931309176 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f433c092e70 con 0x7f433c08f010 2020-03-19T21:19:17.784 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.774+0000 7f4340d6e700 1 --2- 172.21.15.170:0/1931309176 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f433c08f010 0x7f433c0707a0 secure :-1 s=THROTTLE_DONE pgs=309 cs=0 l=1 rx=0x7f432c002f30 tx=0x7f432c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:19:17.784 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.774+0000 7f43397fa700 1 -- 172.21.15.170:0/1931309176 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 4) v1 ==== 29603+0+0 (secure 0 0 0) 0x7f432c0056c0 con 0x7f433c08f010 2020-03-19T21:19:17.784 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.774+0000 7f4340d6e700 1 --2- 172.21.15.170:0/1931309176 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f433c08f010 0x7f433c0707a0 secure :-1 s=THROTTLE_DONE pgs=309 cs=0 l=1 rx=0x7f432c002f30 tx=0x7f432c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:19:17.784 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.774+0000 7f43397fa700 1 --2- 172.21.15.170:0/1931309176 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f431c028230 0x7f431c02a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T21:19:17.785 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.774+0000 7f43397fa700 1 -- 172.21.15.170:0/1931309176 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 3786+0+0 (secure 0 0 0) 0x7f432c034820 con 0x7f433c08f010 2020-03-19T21:19:17.785 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.774+0000 7f433bfff700 1 --2- 172.21.15.170:0/1931309176 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f431c028230 0x7f431c02a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T21:19:17.785 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.774+0000 7f43427d1700 1 -- 172.21.15.170:0/1931309176 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f433c077e30 con 0x7f433c08f010 2020-03-19T21:19:17.785 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.775+0000 7f433bfff700 1 --2- 172.21.15.170:0/1931309176 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f431c028230 0x7f431c02a6f0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f43300012f0 tx=0x7f43300017b0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:19:17.785 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.775+0000 7f433bfff700 1 --2- 172.21.15.170:0/1931309176 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f431c028230 0x7f431c02a6f0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f43300012f0 tx=0x7f43300017b0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:19:17.785 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.775+0000 7f433bfff700 1 --2- 172.21.15.170:0/1931309176 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f431c028230 0x7f431c02a6f0 secure :-1 s=READY pgs=76 cs=0 l=1 rx=0x7f43300012f0 tx=0x7f43300017b0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T21:19:17.786 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.778+0000 7f4340d6e700 1 --2- 172.21.15.170:0/1931309176 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f433c08f010 0x7f433c0707a0 secure :-1 s=THROTTLE_DONE pgs=309 cs=0 l=1 rx=0x7f432c002f30 tx=0x7f432c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:19:17.786 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.778+0000 7f43397fa700 1 -- 172.21.15.170:0/1931309176 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+126772 (secure 0 0 0) 0x7f433c077e30 con 0x7f433c08f010 2020-03-19T21:19:17.917 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.916+0000 7f43427d1700 1 -- 172.21.15.170:0/1931309176 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "config set", "who": "global", "name": "mon_health_to_clog", "value": "false"} v 0) v1 -- 0x7f433c077e30 con 0x7f433c08f010 2020-03-19T21:19:17.921 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.919+0000 7f4340d6e700 1 --2- 172.21.15.170:0/1931309176 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f433c08f010 0x7f433c0707a0 secure :-1 s=THROTTLE_DONE pgs=309 cs=0 l=1 rx=0x7f432c002f30 tx=0x7f432c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:19:17.921 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.919+0000 7f43397fa700 1 -- 172.21.15.170:0/1931309176 <== mon.0 v2:172.21.15.170:3300/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f432c0075f0 con 0x7f433c08f010 2020-03-19T21:19:17.921 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.919+0000 7f4340d6e700 1 --2- 172.21.15.170:0/1931309176 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f433c08f010 0x7f433c0707a0 secure :-1 s=THROTTLE_DONE pgs=309 cs=0 l=1 rx=0x7f432c002f30 tx=0x7f432c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T21:19:17.921 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.919+0000 7f43397fa700 1 -- 172.21.15.170:0/1931309176 <== mon.0 v2:172.21.15.170:3300/0 8 ==== mon_command_ack([{"prefix": "config set", "who": "global", "name": "mon_health_to_clog", "value": "false"}]=0 v3) v1 ==== 123+0+0 (secure 0 0 0) 0x7f433c077e30 con 0x7f433c08f010 2020-03-19T21:19:17.924 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.922+0000 7f43427d1700 1 -- 172.21.15.170:0/1931309176 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f431c028230 msgr2=0x7f431c02a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:19:17.924 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.922+0000 7f43427d1700 1 --2- 172.21.15.170:0/1931309176 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f431c028230 0x7f431c02a6f0 secure :-1 s=READY pgs=76 cs=0 l=1 rx=0x7f43300012f0 tx=0x7f43300017b0).stop 2020-03-19T21:19:17.924 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.922+0000 7f43427d1700 1 -- 172.21.15.170:0/1931309176 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f433c08f010 msgr2=0x7f433c0707a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T21:19:17.925 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.922+0000 7f43427d1700 1 --2- 172.21.15.170:0/1931309176 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f433c08f010 0x7f433c0707a0 secure :-1 s=READY pgs=309 cs=0 l=1 rx=0x7f432c002f30 tx=0x7f432c007a40).stop 2020-03-19T21:19:17.925 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.922+0000 7f43427d1700 1 -- 172.21.15.170:0/1931309176 shutdown_connections 2020-03-19T21:19:17.925 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.922+0000 7f43427d1700 1 --2- 172.21.15.170:0/1931309176 >> [v2:172.21.15.170:6824/20154,v1:172.21.15.170:6825/20154] conn(0x7f431c028230 0x7f431c02a6f0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:19:17.925 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.922+0000 7f43427d1700 1 --2- 172.21.15.170:0/1931309176 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f433c08f010 0x7f433c0707a0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T21:19:17.926 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.922+0000 7f43427d1700 1 -- 172.21.15.170:0/1931309176 >> 172.21.15.170:0/1931309176 conn(0x7f433c095090 msgr2=0x7f433c066680 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T21:19:17.926 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.922+0000 7f43427d1700 1 -- 172.21.15.170:0/1931309176 shutdown_connections 2020-03-19T21:19:17.926 INFO:teuthology.orchestra.run.smithi170.stderr:2020-03-19T21:19:17.922+0000 7f43427d1700 1 -- 172.21.15.170:0/1931309176 wait complete. 2020-03-19T21:19:17.933 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/contextutil.py", line 34, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-sage-testing-2020-03-19-0855/qa/tasks/ceph.py", line 1892, in task yield File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/run_tasks.py", line 89, in run_tasks manager.__enter__() File "/usr/lib/python2.7/contextlib.py", line 17, in __enter__ return self.gen.next() File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-sage-testing-2020-03-19-0855/qa/tasks/ceph_fuse.py", line 140, in task info["mount"].mount(mountpoint=mountpoint, mount_path=mount_path) File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-sage-testing-2020-03-19-0855/qa/tasks/cephfs/fuse_mount.py", line 33, in mount return self._mount(mount_path, mount_fs_name, mount_options) File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-sage-testing-2020-03-19-0855/qa/tasks/cephfs/fuse_mount.py", line 163, in _mount self.gather_mount_info() File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-sage-testing-2020-03-19-0855/qa/tasks/cephfs/fuse_mount.py", line 166, in gather_mount_info status = self.admin_socket(['status']) File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-sage-testing-2020-03-19-0855/qa/tasks/cephfs/fuse_mount.py", line 449, in admin_socket ], stdout=StringIO(), timeout=(15*60)) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/remote.py", line 198, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 433, in run r.wait() File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 158, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 180, in _raise_for_status node=self.hostname, label=self.label CommandFailedError: Command failed on smithi170 with status 1: 'sudo python3 -c \'\'"\'"\'\nimport glob\nimport re\nimport os\nimport subprocess\n\ndef find_socket(client_name):\n asok_path = "/var/run/ceph/ceph-client.1.*.asok"\n files = glob.glob(asok_path)\n\n # Given a non-glob path, it better be there\n if "*" not in asok_path:\n assert(len(files) == 1)\n return files[0]\n\n for f in files:\n pid = re.match(".*\\.(\\d+)\\.asok$", f).group(1)\n if os.path.exists("/proc/{0}".format(pid)):\n return f\n raise RuntimeError("Client socket {0} not found".format(client_name))\n\nprint(find_socket("client.1"))\n\'"\'"\'\'' 2020-03-19T21:19:17.941 INFO:teuthology.misc:Shutting down mds daemons... 2020-03-19T21:19:17.941 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2020-03-19T21:19:17.942 INFO:teuthology.orchestra.run:waiting for 300 2020-03-19T21:19:17.949 INFO:tasks.ceph.mds.a:Stopped 2020-03-19T21:19:17.949 INFO:teuthology.misc:Shutting down osd daemons... 2020-03-19T21:19:17.949 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2020-03-19T21:19:17.950 INFO:teuthology.orchestra.run:waiting for 300 2020-03-19T21:19:18.013 INFO:tasks.ceph.osd.1:Stopped 2020-03-19T21:19:18.013 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2020-03-19T21:19:18.013 INFO:teuthology.orchestra.run:waiting for 300 2020-03-19T21:19:18.050 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~0s 2020-03-19T21:19:18.078 INFO:tasks.ceph.osd.0:Stopped 2020-03-19T21:19:18.078 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2020-03-19T21:19:18.078 INFO:teuthology.orchestra.run:waiting for 300 2020-03-19T21:19:18.142 INFO:tasks.ceph.osd.2:Stopped 2020-03-19T21:19:18.142 INFO:teuthology.misc:Shutting down mgr daemons... 2020-03-19T21:19:18.142 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2020-03-19T21:19:18.142 INFO:teuthology.orchestra.run:waiting for 300 2020-03-19T21:19:18.198 INFO:tasks.ceph.mgr.x:Stopped 2020-03-19T21:19:18.198 INFO:teuthology.misc:Shutting down mon daemons... 2020-03-19T21:19:18.198 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2020-03-19T21:19:18.198 INFO:teuthology.orchestra.run:waiting for 300 2020-03-19T21:19:18.247 INFO:tasks.ceph.mon.a:Stopped 2020-03-19T21:19:18.247 INFO:tasks.ceph:Checking cluster log for badness... 2020-03-19T21:19:18.247 INFO:teuthology.orchestra.run.smithi170:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v 'overall HEALTH_' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | head -n 1 2020-03-19T21:19:18.317 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi170.front.sepia.ceph.com 2020-03-19T21:19:18.317 INFO:teuthology.orchestra.run.smithi170:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2020-03-19T21:19:18.585 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi170.front.sepia.ceph.com 2020-03-19T21:19:18.585 INFO:teuthology.orchestra.run.smithi170:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2020-03-19T21:19:18.685 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi170.front.sepia.ceph.com 2020-03-19T21:19:18.686 INFO:teuthology.orchestra.run.smithi170:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2020-03-19T21:19:18.763 INFO:tasks.ceph:Archiving mon data... 2020-03-19T21:19:18.764 DEBUG:teuthology.misc:Transferring archived files from smithi170:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/sage-2020-03-19_17:25:48-rados-wip-sage-testing-2020-03-19-0855-distro-basic-smithi/4868524/data/mon.a.tgz 2020-03-19T21:19:18.764 INFO:teuthology.orchestra.run.smithi170:> mktemp 2020-03-19T21:19:18.778 INFO:teuthology.orchestra.run.smithi170.stdout:/tmp/tmp.uIADe64dcC 2020-03-19T21:19:18.778 INFO:teuthology.orchestra.run.smithi170:> sudo tar cz -f /tmp/tmp.uIADe64dcC -C /var/lib/ceph/mon/ceph-a -- . 2020-03-19T21:19:18.906 INFO:teuthology.orchestra.run.smithi170:> sudo chmod 0666 /tmp/tmp.uIADe64dcC 2020-03-19T21:19:18.989 DEBUG:teuthology.orchestra.remote:smithi170:/tmp/tmp.uIADe64dcC is 269KB 2020-03-19T21:19:19.082 INFO:teuthology.orchestra.run.smithi170:> rm -fr /tmp/tmp.uIADe64dcC 2020-03-19T21:19:19.095 INFO:tasks.ceph:Cleaning ceph cluster... 2020-03-19T21:19:19.096 INFO:teuthology.orchestra.run.smithi170:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2020-03-19T21:19:19.161 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2020-03-19T21:19:19.161 INFO:teuthology.orchestra.run.smithi170:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2020-03-19T21:19:19.228 INFO:teuthology.orchestra.run.smithi170.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2020-03-19T21:19:19.233 INFO:tasks.ceph:Archiving crash dumps... 2020-03-19T21:19:19.235 DEBUG:teuthology.misc:Transferring archived files from smithi170:/var/lib/ceph/crash to /home/teuthworker/archive/sage-2020-03-19_17:25:48-rados-wip-sage-testing-2020-03-19-0855-distro-basic-smithi/4868524/remote/smithi170/crash 2020-03-19T21:19:19.236 INFO:teuthology.orchestra.run.smithi170:> sudo tar cz -f - -C /var/lib/ceph/crash -- . 2020-03-19T21:19:19.303 INFO:tasks.ceph:Compressing logs... 2020-03-19T21:19:19.303 INFO:teuthology.orchestra.run.smithi170:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2020-03-19T21:19:19.777 INFO:tasks.ceph:Archiving logs... 2020-03-19T21:19:19.777 DEBUG:teuthology.misc:Transferring archived files from smithi170:/var/log/ceph to /home/teuthworker/archive/sage-2020-03-19_17:25:48-rados-wip-sage-testing-2020-03-19-0855-distro-basic-smithi/4868524/remote/smithi170/log 2020-03-19T21:19:19.778 INFO:teuthology.orchestra.run.smithi170:> sudo tar cz -f - -C /var/log/ceph -- . 2020-03-19T21:19:19.938 DEBUG:teuthology.run_tasks:Unwinding manager install 2020-03-19T21:19:19.960 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/contextutil.py", line 34, in nested yield vars File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/task/install/__init__.py", line 599, in task yield File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/run_tasks.py", line 89, in run_tasks manager.__enter__() File "/usr/lib/python2.7/contextlib.py", line 17, in __enter__ return self.gen.next() File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-sage-testing-2020-03-19-0855/qa/tasks/ceph_fuse.py", line 140, in task info["mount"].mount(mountpoint=mountpoint, mount_path=mount_path) File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-sage-testing-2020-03-19-0855/qa/tasks/cephfs/fuse_mount.py", line 33, in mount return self._mount(mount_path, mount_fs_name, mount_options) File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-sage-testing-2020-03-19-0855/qa/tasks/cephfs/fuse_mount.py", line 163, in _mount self.gather_mount_info() File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-sage-testing-2020-03-19-0855/qa/tasks/cephfs/fuse_mount.py", line 166, in gather_mount_info status = self.admin_socket(['status']) File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-sage-testing-2020-03-19-0855/qa/tasks/cephfs/fuse_mount.py", line 449, in admin_socket ], stdout=StringIO(), timeout=(15*60)) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/remote.py", line 198, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 433, in run r.wait() File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 158, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 180, in _raise_for_status node=self.hostname, label=self.label CommandFailedError: Command failed on smithi170 with status 1: 'sudo python3 -c \'\'"\'"\'\nimport glob\nimport re\nimport os\nimport subprocess\n\ndef find_socket(client_name):\n asok_path = "/var/run/ceph/ceph-client.1.*.asok"\n files = glob.glob(asok_path)\n\n # Given a non-glob path, it better be there\n if "*" not in asok_path:\n assert(len(files) == 1)\n return files[0]\n\n for f in files:\n pid = re.match(".*\\.(\\d+)\\.asok$", f).group(1)\n if os.path.exists("/proc/{0}".format(pid)):\n return f\n raise RuntimeError("Client socket {0} not found".format(client_name))\n\nprint(find_socket("client.1"))\n\'"\'"\'\'' 2020-03-19T21:19:19.960 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits... 2020-03-19T21:19:19.961 INFO:teuthology.orchestra.run.smithi170:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits 2020-03-19T21:19:19.988 INFO:teuthology.orchestra.run.smithi170:> sudo lsb_release -is 2020-03-19T21:19:20.063 INFO:teuthology.orchestra.run.smithi170.stdout:RedHatEnterprise 2020-03-19T21:19:20.064 DEBUG:teuthology.misc:System to be installed: RedHatEnterprise 2020-03-19T21:19:20.065 INFO:teuthology.task.install.rpm:Removing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on rpm system. 2020-03-19T21:19:20.065 INFO:teuthology.orchestra.run.smithi170:> 2020-03-19T21:19:20.065 INFO:teuthology.orchestra.run.smithi170:> for d in ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd ; do 2020-03-19T21:19:20.065 INFO:teuthology.orchestra.run.smithi170:> sudo yum -y remove $d || true 2020-03-19T21:19:20.065 INFO:teuthology.orchestra.run.smithi170:> done 2020-03-19T21:19:20.456 INFO:teuthology.orchestra.run.smithi170.stdout:Updating Subscription Management repositories. 2020-03-19T21:19:21.132 INFO:teuthology.orchestra.run.smithi170.stderr:Modular dependency problem: 2020-03-19T21:19:21.132 INFO:teuthology.orchestra.run.smithi170.stderr: 2020-03-19T21:19:21.133 INFO:teuthology.orchestra.run.smithi170.stderr: Problem: conflicting requests 2020-03-19T21:19:21.133 INFO:teuthology.orchestra.run.smithi170.stderr: - nothing provides module(perl:5.26) needed by module perl-YAML:1.24:8010020190322131811:a5949e2e-0.x86_64 2020-03-19T21:19:21.167 INFO:teuthology.orchestra.run.smithi170.stdout:Dependencies resolved. 2020-03-19T21:19:21.167 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2020-03-19T21:19:21.168 INFO:teuthology.orchestra.run.smithi170.stdout: Package Arch Version Repository Size 2020-03-19T21:19:21.168 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2020-03-19T21:19:21.168 INFO:teuthology.orchestra.run.smithi170.stdout:Removing: 2020-03-19T21:19:21.168 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-radosgw x86_64 2:15.1.1-234.g787db24.el8 @ceph 40 M 2020-03-19T21:19:21.168 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:19:21.168 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction Summary 2020-03-19T21:19:21.168 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2020-03-19T21:19:21.169 INFO:teuthology.orchestra.run.smithi170.stdout:Remove 1 Package 2020-03-19T21:19:21.169 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:19:21.169 INFO:teuthology.orchestra.run.smithi170.stdout:Freed space: 40 M 2020-03-19T21:19:21.169 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction check 2020-03-19T21:19:21.171 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction check succeeded. 2020-03-19T21:19:21.171 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction test 2020-03-19T21:19:21.199 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction test succeeded. 2020-03-19T21:19:21.199 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction 2020-03-19T21:19:21.264 INFO:teuthology.orchestra.run.smithi170.stdout: Preparing : 1/1 2020-03-19T21:19:21.295 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-radosgw-2:15.1.1-234.g787db24.el8.x86_64 1/1 2020-03-19T21:19:21.295 INFO:teuthology.orchestra.run.smithi170.stdout: Erasing : ceph-radosgw-2:15.1.1-234.g787db24.el8.x86_64 1/1 2020-03-19T21:19:21.809 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-radosgw-2:15.1.1-234.g787db24.el8.x86_64 1/1 2020-03-19T21:19:23.051 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a has been restored 2020-03-19T21:19:23.837 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-radosgw-2:15.1.1-234.g787db24.el8.x86_64 1/1 2020-03-19T21:19:23.837 INFO:teuthology.orchestra.run.smithi170.stdout:Installed products updated. 2020-03-19T21:19:24.016 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:19:24.017 INFO:teuthology.orchestra.run.smithi170.stdout:Removed: 2020-03-19T21:19:24.017 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-radosgw-2:15.1.1-234.g787db24.el8.x86_64 2020-03-19T21:19:24.017 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:19:24.017 INFO:teuthology.orchestra.run.smithi170.stdout:Complete! 2020-03-19T21:19:24.541 INFO:teuthology.orchestra.run.smithi170.stdout:Updating Subscription Management repositories. 2020-03-19T21:19:25.221 INFO:teuthology.orchestra.run.smithi170.stderr:Modular dependency problem: 2020-03-19T21:19:25.221 INFO:teuthology.orchestra.run.smithi170.stderr: 2020-03-19T21:19:25.221 INFO:teuthology.orchestra.run.smithi170.stderr: Problem: conflicting requests 2020-03-19T21:19:25.221 INFO:teuthology.orchestra.run.smithi170.stderr: - nothing provides module(perl:5.26) needed by module perl-YAML:1.24:8010020190322131811:a5949e2e-0.x86_64 2020-03-19T21:19:25.256 INFO:teuthology.orchestra.run.smithi170.stdout:Dependencies resolved. 2020-03-19T21:19:25.258 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2020-03-19T21:19:25.258 INFO:teuthology.orchestra.run.smithi170.stdout: Package Arch Version Repository Size 2020-03-19T21:19:25.258 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2020-03-19T21:19:25.258 INFO:teuthology.orchestra.run.smithi170.stdout:Removing: 2020-03-19T21:19:25.259 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-test x86_64 2:15.1.1-234.g787db24.el8 2020-03-19T21:19:25.259 INFO:teuthology.orchestra.run.smithi170.stdout: @ceph 182 M 2020-03-19T21:19:25.259 INFO:teuthology.orchestra.run.smithi170.stdout:Removing unused dependencies: 2020-03-19T21:19:25.259 INFO:teuthology.orchestra.run.smithi170.stdout: jq x86_64 1.5-12.el8 @rhel-8-for-x86_64-appstream-rpms 357 k 2020-03-19T21:19:25.259 INFO:teuthology.orchestra.run.smithi170.stdout: oniguruma x86_64 6.8.2-1.el8 @rhel-8-for-x86_64-appstream-rpms 692 k 2020-03-19T21:19:25.259 INFO:teuthology.orchestra.run.smithi170.stdout: socat x86_64 1.7.3.2-6.el8 @rhel-8-for-x86_64-appstream-rpms 1.1 M 2020-03-19T21:19:25.259 INFO:teuthology.orchestra.run.smithi170.stdout: xmlstarlet x86_64 1.6.1-11.el8 @epel 258 k 2020-03-19T21:19:25.259 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:19:25.260 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction Summary 2020-03-19T21:19:25.260 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2020-03-19T21:19:25.260 INFO:teuthology.orchestra.run.smithi170.stdout:Remove 5 Packages 2020-03-19T21:19:25.260 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:19:25.260 INFO:teuthology.orchestra.run.smithi170.stdout:Freed space: 184 M 2020-03-19T21:19:25.261 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction check 2020-03-19T21:19:25.263 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction check succeeded. 2020-03-19T21:19:25.263 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction test 2020-03-19T21:19:25.309 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction test succeeded. 2020-03-19T21:19:25.310 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction 2020-03-19T21:19:25.447 INFO:teuthology.orchestra.run.smithi170.stdout: Preparing : 1/1 2020-03-19T21:19:25.667 INFO:teuthology.orchestra.run.smithi170.stdout: Erasing : ceph-test-2:15.1.1-234.g787db24.el8.x86_64 1/5 2020-03-19T21:19:25.817 INFO:teuthology.orchestra.run.smithi170.stdout: Erasing : jq-1.5-12.el8.x86_64 2/5 2020-03-19T21:19:25.817 INFO:teuthology.orchestra.run.smithi170.stdout: Erasing : oniguruma-6.8.2-1.el8.x86_64 3/5 2020-03-19T21:19:26.035 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: oniguruma-6.8.2-1.el8.x86_64 3/5 2020-03-19T21:19:26.217 INFO:teuthology.orchestra.run.smithi170.stdout: Erasing : socat-1.7.3.2-6.el8.x86_64 4/5 2020-03-19T21:19:26.328 INFO:teuthology.orchestra.run.smithi170.stdout: Erasing : xmlstarlet-1.6.1-11.el8.x86_64 5/5 2020-03-19T21:19:26.538 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: xmlstarlet-1.6.1-11.el8.x86_64 5/5 2020-03-19T21:19:26.538 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-test-2:15.1.1-234.g787db24.el8.x86_64 1/5 2020-03-19T21:19:26.539 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : jq-1.5-12.el8.x86_64 2/5 2020-03-19T21:19:26.539 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : oniguruma-6.8.2-1.el8.x86_64 3/5 2020-03-19T21:19:26.539 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : socat-1.7.3.2-6.el8.x86_64 4/5 2020-03-19T21:19:28.702 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : xmlstarlet-1.6.1-11.el8.x86_64 5/5 2020-03-19T21:19:28.703 INFO:teuthology.orchestra.run.smithi170.stdout:Installed products updated. 2020-03-19T21:19:28.949 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:19:28.949 INFO:teuthology.orchestra.run.smithi170.stdout:Removed: 2020-03-19T21:19:28.949 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-test-2:15.1.1-234.g787db24.el8.x86_64 jq-1.5-12.el8.x86_64 2020-03-19T21:19:28.950 INFO:teuthology.orchestra.run.smithi170.stdout: oniguruma-6.8.2-1.el8.x86_64 socat-1.7.3.2-6.el8.x86_64 2020-03-19T21:19:28.950 INFO:teuthology.orchestra.run.smithi170.stdout: xmlstarlet-1.6.1-11.el8.x86_64 2020-03-19T21:19:28.950 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:19:28.950 INFO:teuthology.orchestra.run.smithi170.stdout:Complete! 2020-03-19T21:19:29.524 INFO:teuthology.orchestra.run.smithi170.stdout:Updating Subscription Management repositories. 2020-03-19T21:19:30.050 INFO:teuthology.orchestra.run.smithi170.stderr:Modular dependency problem: 2020-03-19T21:19:30.050 INFO:teuthology.orchestra.run.smithi170.stderr: 2020-03-19T21:19:30.050 INFO:teuthology.orchestra.run.smithi170.stderr: Problem: conflicting requests 2020-03-19T21:19:30.050 INFO:teuthology.orchestra.run.smithi170.stderr: - nothing provides module(perl:5.26) needed by module perl-YAML:1.24:8010020190322131811:a5949e2e-0.x86_64 2020-03-19T21:19:30.085 INFO:teuthology.orchestra.run.smithi170.stdout:Dependencies resolved. 2020-03-19T21:19:30.086 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2020-03-19T21:19:30.086 INFO:teuthology.orchestra.run.smithi170.stdout: Package Architecture Version Repository Size 2020-03-19T21:19:30.087 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2020-03-19T21:19:30.087 INFO:teuthology.orchestra.run.smithi170.stdout:Removing: 2020-03-19T21:19:30.087 INFO:teuthology.orchestra.run.smithi170.stdout: ceph x86_64 2:15.1.1-234.g787db24.el8 @ceph 0 2020-03-19T21:19:30.087 INFO:teuthology.orchestra.run.smithi170.stdout:Removing unused dependencies: 2020-03-19T21:19:30.087 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mds x86_64 2:15.1.1-234.g787db24.el8 @ceph 7.3 M 2020-03-19T21:19:30.087 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mon x86_64 2:15.1.1-234.g787db24.el8 @ceph 15 M 2020-03-19T21:19:30.087 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-osd x86_64 2:15.1.1-234.g787db24.el8 @ceph 59 M 2020-03-19T21:19:30.088 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:19:30.088 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction Summary 2020-03-19T21:19:30.088 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2020-03-19T21:19:30.088 INFO:teuthology.orchestra.run.smithi170.stdout:Remove 4 Packages 2020-03-19T21:19:30.088 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:19:30.088 INFO:teuthology.orchestra.run.smithi170.stdout:Freed space: 81 M 2020-03-19T21:19:30.089 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction check 2020-03-19T21:19:30.090 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction check succeeded. 2020-03-19T21:19:30.090 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction test 2020-03-19T21:19:30.118 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction test succeeded. 2020-03-19T21:19:30.119 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction 2020-03-19T21:19:30.230 INFO:teuthology.orchestra.run.smithi170.stdout: Preparing : 1/1 2020-03-19T21:19:30.236 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-2:15.1.1-234.g787db24.el8.x86_64 1/1 2020-03-19T21:19:30.341 INFO:teuthology.orchestra.run.smithi170.stdout: Erasing : ceph-2:15.1.1-234.g787db24.el8.x86_64 1/4 2020-03-19T21:19:30.359 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-mds-2:15.1.1-234.g787db24.el8.x86_64 2/4 2020-03-19T21:19:30.359 INFO:teuthology.orchestra.run.smithi170.stdout: Erasing : ceph-mds-2:15.1.1-234.g787db24.el8.x86_64 2/4 2020-03-19T21:19:30.600 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-mds-2:15.1.1-234.g787db24.el8.x86_64 2/4 2020-03-19T21:19:30.619 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-mon-2:15.1.1-234.g787db24.el8.x86_64 3/4 2020-03-19T21:19:30.619 INFO:teuthology.orchestra.run.smithi170.stdout: Erasing : ceph-mon-2:15.1.1-234.g787db24.el8.x86_64 3/4 2020-03-19T21:19:30.791 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-mon-2:15.1.1-234.g787db24.el8.x86_64 3/4 2020-03-19T21:19:30.820 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-osd-2:15.1.1-234.g787db24.el8.x86_64 4/4 2020-03-19T21:19:30.820 INFO:teuthology.orchestra.run.smithi170.stdout: Erasing : ceph-osd-2:15.1.1-234.g787db24.el8.x86_64 4/4 2020-03-19T21:19:31.185 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-osd-2:15.1.1-234.g787db24.el8.x86_64 4/4 2020-03-19T21:19:31.185 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-2:15.1.1-234.g787db24.el8.x86_64 1/4 2020-03-19T21:19:31.186 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mds-2:15.1.1-234.g787db24.el8.x86_64 2/4 2020-03-19T21:19:31.186 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mon-2:15.1.1-234.g787db24.el8.x86_64 3/4 2020-03-19T21:19:33.354 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-osd-2:15.1.1-234.g787db24.el8.x86_64 4/4 2020-03-19T21:19:33.354 INFO:teuthology.orchestra.run.smithi170.stdout:Installed products updated. 2020-03-19T21:19:33.548 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:19:33.548 INFO:teuthology.orchestra.run.smithi170.stdout:Removed: 2020-03-19T21:19:33.549 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-2:15.1.1-234.g787db24.el8.x86_64 2020-03-19T21:19:33.549 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mds-2:15.1.1-234.g787db24.el8.x86_64 2020-03-19T21:19:33.549 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mon-2:15.1.1-234.g787db24.el8.x86_64 2020-03-19T21:19:33.549 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-osd-2:15.1.1-234.g787db24.el8.x86_64 2020-03-19T21:19:33.549 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:19:33.549 INFO:teuthology.orchestra.run.smithi170.stdout:Complete! 2020-03-19T21:19:34.112 INFO:teuthology.orchestra.run.smithi170.stdout:Updating Subscription Management repositories. 2020-03-19T21:19:34.702 INFO:teuthology.orchestra.run.smithi170.stderr:Modular dependency problem: 2020-03-19T21:19:34.702 INFO:teuthology.orchestra.run.smithi170.stderr: 2020-03-19T21:19:34.702 INFO:teuthology.orchestra.run.smithi170.stderr: Problem: conflicting requests 2020-03-19T21:19:34.702 INFO:teuthology.orchestra.run.smithi170.stderr: - nothing provides module(perl:5.26) needed by module perl-YAML:1.24:8010020190322131811:a5949e2e-0.x86_64 2020-03-19T21:19:34.745 INFO:teuthology.orchestra.run.smithi170.stdout:Dependencies resolved. 2020-03-19T21:19:34.759 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2020-03-19T21:19:34.759 INFO:teuthology.orchestra.run.smithi170.stdout: Package ArchVersion Repository Size 2020-03-19T21:19:34.759 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2020-03-19T21:19:34.759 INFO:teuthology.orchestra.run.smithi170.stdout:Removing: 2020-03-19T21:19:34.759 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-base x86_642:15.1.1-234.g787db24.el8 @ceph 26 M 2020-03-19T21:19:34.759 INFO:teuthology.orchestra.run.smithi170.stdout:Removing dependent packages: 2020-03-19T21:19:34.760 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-rook noarch2:15.1.1-234.g787db24.el8 @ceph-noarch 107 k 2020-03-19T21:19:34.760 INFO:teuthology.orchestra.run.smithi170.stdout: rbd-mirror x86_642:15.1.1-234.g787db24.el8 @ceph 13 M 2020-03-19T21:19:34.760 INFO:teuthology.orchestra.run.smithi170.stdout:Removing unused dependencies: 2020-03-19T21:19:34.760 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-common x86_642:15.1.1-234.g787db24.el8 @ceph 79 M 2020-03-19T21:19:34.760 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-grafana-dashboards noarch2:15.1.1-234.g787db24.el8 @ceph-noarch 201 k 2020-03-19T21:19:34.760 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr x86_642:15.1.1-234.g787db24.el8 @ceph 5.3 M 2020-03-19T21:19:34.760 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-cephadm noarch2:15.1.1-234.g787db24.el8 @ceph-noarch 142 k 2020-03-19T21:19:34.761 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-dashboard noarch2:15.1.1-234.g787db24.el8 @ceph-noarch 17 M 2020-03-19T21:19:34.761 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-diskprediction-local noarch2:15.1.1-234.g787db24.el8 @ceph-noarch 66 M 2020-03-19T21:19:34.761 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-modules-core noarch2:15.1.1-234.g787db24.el8 @ceph-noarch 872 k 2020-03-19T21:19:34.761 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-selinux x86_642:15.1.1-234.g787db24.el8 @ceph 131 k 2020-03-19T21:19:34.761 INFO:teuthology.orchestra.run.smithi170.stdout: gperftools-libs x86_642.7-6.el8 @epel 1.7 M 2020-03-19T21:19:34.761 INFO:teuthology.orchestra.run.smithi170.stdout: leveldb x86_641.21-2.el8 @lab-extras 482 k 2020-03-19T21:19:34.761 INFO:teuthology.orchestra.run.smithi170.stdout: libgfortran x86_648.3.1-4.5.el8 @rhel-8-for-x86_64-baseos-rpms 2.5 M 2020-03-19T21:19:34.762 INFO:teuthology.orchestra.run.smithi170.stdout: liboath x86_642.6.2-3.el8 @epel 125 k 2020-03-19T21:19:34.762 INFO:teuthology.orchestra.run.smithi170.stdout: libquadmath x86_648.3.1-4.5.el8 @rhel-8-for-x86_64-baseos-rpms 299 k 2020-03-19T21:19:34.762 INFO:teuthology.orchestra.run.smithi170.stdout: librabbitmq x86_640.9.0-1.el8 @rhel-8-for-x86_64-baseos-rpms 95 k 2020-03-19T21:19:34.762 INFO:teuthology.orchestra.run.smithi170.stdout: libradosstriper1 x86_642:15.1.1-234.g787db24.el8 @ceph 1.4 M 2020-03-19T21:19:34.762 INFO:teuthology.orchestra.run.smithi170.stdout: librdkafka x86_640.11.4-1.el8 @rhel-8-for-x86_64-appstream-rpms 1.1 M 2020-03-19T21:19:34.762 INFO:teuthology.orchestra.run.smithi170.stdout: librgw2 x86_642:15.1.1-234.g787db24.el8 @ceph 25 M 2020-03-19T21:19:34.762 INFO:teuthology.orchestra.run.smithi170.stdout: libunwind x86_641.3.1-3.el8 @epel 205 k 2020-03-19T21:19:34.763 INFO:teuthology.orchestra.run.smithi170.stdout: libxslt x86_641.1.32-3.el8 @rhel-8-for-x86_64-baseos-rpms 743 k 2020-03-19T21:19:34.763 INFO:teuthology.orchestra.run.smithi170.stdout: openblas x86_640.3.3-2.el8 @rhel-8-for-x86_64-appstream-rpms 31 M 2020-03-19T21:19:34.763 INFO:teuthology.orchestra.run.smithi170.stdout: openblas-threads x86_640.3.3-2.el8 @rhel-8-for-x86_64-appstream-rpms 32 M 2020-03-19T21:19:34.763 INFO:teuthology.orchestra.run.smithi170.stdout: python3-apipkg noarch1.5-6.el8 @epel 24 k 2020-03-19T21:19:34.763 INFO:teuthology.orchestra.run.smithi170.stdout: python3-bcrypt x86_643.1.6-2.el8.1 @epel 78 k 2020-03-19T21:19:34.763 INFO:teuthology.orchestra.run.smithi170.stdout: python3-beautifulsoup4 noarch4.6.3-2.el8.1 @epel 817 k 2020-03-19T21:19:34.763 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cachetools noarch3.1.1-4.el8 @epel 71 k 2020-03-19T21:19:34.764 INFO:teuthology.orchestra.run.smithi170.stdout: python3-ceph-argparse x86_642:15.1.1-234.g787db24.el8 @ceph 113 k 2020-03-19T21:19:34.764 INFO:teuthology.orchestra.run.smithi170.stdout: python3-ceph-common x86_642:15.1.1-234.g787db24.el8 @ceph 216 k 2020-03-19T21:19:34.764 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cephfs x86_642:15.1.1-234.g787db24.el8 @ceph 520 k 2020-03-19T21:19:34.764 INFO:teuthology.orchestra.run.smithi170.stdout: python3-certifi noarch2018.10.15-7.el8 @epel 8.2 k 2020-03-19T21:19:34.764 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cheroot noarch8.2.1-1.el8 @copr:copr.fedorainfracloud.org:ktdreyer:ceph-el8 531 k 2020-03-19T21:19:34.764 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cherrypy noarch18.4.0-1.el8 @copr:copr.fedorainfracloud.org:ktdreyer:ceph-el8 1.1 M 2020-03-19T21:19:34.765 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cssselect noarch0.9.2-10.el8 @epel 142 k 2020-03-19T21:19:34.765 INFO:teuthology.orchestra.run.smithi170.stdout: python3-execnet noarch1.7.1-1.el8.2 @copr:copr.fedorainfracloud.org:ktdreyer:ceph-el8 1.2 M 2020-03-19T21:19:34.765 INFO:teuthology.orchestra.run.smithi170.stdout: python3-google-auth noarch1:1.1.1-10.el8 @epel 306 k 2020-03-19T21:19:34.765 INFO:teuthology.orchestra.run.smithi170.stdout: python3-html5lib noarch1:0.999999999-6.el8 @rhel-8-for-x86_64-appstream-rpms 1.1 M 2020-03-19T21:19:34.766 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco noarch6.2-6.el8 @copr:copr.fedorainfracloud.org:ktdreyer:ceph-el8 4.0 k 2020-03-19T21:19:34.766 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco-functools noarch2.0-4.el8 @copr:copr.fedorainfracloud.org:ktdreyer:ceph-el8 28 k 2020-03-19T21:19:34.767 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jsonpatch noarch1.21-2.el8 @rhel-8-for-x86_64-appstream-rpms 55 k 2020-03-19T21:19:34.767 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jsonpointer noarch1.10-11.el8 @rhel-8-for-x86_64-appstream-rpms 32 k 2020-03-19T21:19:34.767 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jwt noarch1.6.1-2.el8 @rhel-8-for-x86_64-baseos-rpms 88 k 2020-03-19T21:19:34.767 INFO:teuthology.orchestra.run.smithi170.stdout: python3-kubernetes noarch10.0.1-2.el8 @epel 20 M 2020-03-19T21:19:34.767 INFO:teuthology.orchestra.run.smithi170.stdout: python3-logutils noarch0.3.5-11.el8 @copr:copr.fedorainfracloud.org:ktdreyer:ceph-el8 126 k 2020-03-19T21:19:34.768 INFO:teuthology.orchestra.run.smithi170.stdout: python3-lxml x86_644.2.3-1.el8 @rhel-8-for-x86_64-appstream-rpms 4.8 M 2020-03-19T21:19:34.768 INFO:teuthology.orchestra.run.smithi170.stdout: python3-mako noarch1.0.6-13.el8 @rhel-8-for-x86_64-appstream-rpms 540 k 2020-03-19T21:19:34.768 INFO:teuthology.orchestra.run.smithi170.stdout: python3-markupsafe x86_640.23-19.el8 @rhel-8-for-x86_64-appstream-rpms 76 k 2020-03-19T21:19:34.768 INFO:teuthology.orchestra.run.smithi170.stdout: python3-more-itertools noarch7.2.0-3.el8 @copr:copr.fedorainfracloud.org:ktdreyer:ceph-el8 302 k 2020-03-19T21:19:34.768 INFO:teuthology.orchestra.run.smithi170.stdout: python3-numpy x86_641:1.14.3-9.el8 @rhel-8-for-x86_64-appstream-rpms 16 M 2020-03-19T21:19:34.768 INFO:teuthology.orchestra.run.smithi170.stdout: python3-numpy-f2py x86_641:1.14.3-9.el8 @rhel-8-for-x86_64-appstream-rpms 946 k 2020-03-19T21:19:34.768 INFO:teuthology.orchestra.run.smithi170.stdout: python3-oauthlib noarch2.1.0-1.el8 @rhel-8-for-x86_64-baseos-rpms 737 k 2020-03-19T21:19:34.769 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pecan noarch1.3.2-9.el8 @copr:copr.fedorainfracloud.org:ktdreyer:ceph-el8 1.3 M 2020-03-19T21:19:34.769 INFO:teuthology.orchestra.run.smithi170.stdout: python3-portend noarch2.6-1.el8 @copr:copr.fedorainfracloud.org:ktdreyer:ceph-el8 19 k 2020-03-19T21:19:34.769 INFO:teuthology.orchestra.run.smithi170.stdout: python3-prettytable noarch0.7.2-14.el8 @rhel-8-for-x86_64-appstream-rpms 167 k 2020-03-19T21:19:34.769 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pyasn1 noarch0.3.7-6.el8 @rhel-8-for-x86_64-appstream-rpms 522 k 2020-03-19T21:19:34.769 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pyasn1-modules noarch0.3.7-6.el8 @rhel-8-for-x86_64-appstream-rpms 603 k 2020-03-19T21:19:34.769 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pytz noarch2017.2-9.el8 @rhel-8-for-x86_64-appstream-rpms 175 k 2020-03-19T21:19:34.769 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rados x86_642:15.1.1-234.g787db24.el8 @ceph 1.1 M 2020-03-19T21:19:34.770 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rbd x86_642:15.1.1-234.g787db24.el8 @ceph 1.1 M 2020-03-19T21:19:34.770 INFO:teuthology.orchestra.run.smithi170.stdout: python3-remoto noarch1.1.4-4.el8 @copr:copr.fedorainfracloud.org:ktdreyer:ceph-el8 97 k 2020-03-19T21:19:34.770 INFO:teuthology.orchestra.run.smithi170.stdout: python3-repoze-lru noarch0.7-6.el8 @copr:copr.fedorainfracloud.org:ktdreyer:ceph-el8 84 k 2020-03-19T21:19:34.770 INFO:teuthology.orchestra.run.smithi170.stdout: python3-requests-oauthlib noarch1.0.0-1.el8 @rhel-8-for-x86_64-baseos-rpms 107 k 2020-03-19T21:19:34.770 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rgw x86_642:15.1.1-234.g787db24.el8 @ceph 293 k 2020-03-19T21:19:34.770 INFO:teuthology.orchestra.run.smithi170.stdout: python3-routes noarch2.4.1-12.el8 @copr:copr.fedorainfracloud.org:ktdreyer:ceph-el8 454 k 2020-03-19T21:19:34.771 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rsa noarch3.4.2-12.el8 @epel 240 k 2020-03-19T21:19:34.771 INFO:teuthology.orchestra.run.smithi170.stdout: python3-scipy x86_641.0.0-20.module+el8.1.0+3334+5cb623d7 @rhel-8-for-x86_64-appstream-rpms 56 M 2020-03-19T21:19:34.771 INFO:teuthology.orchestra.run.smithi170.stdout: python3-simplegeneric noarch0.8.1-17.el8 @copr:copr.fedorainfracloud.org:ktdreyer:ceph-el8 24 k 2020-03-19T21:19:34.771 INFO:teuthology.orchestra.run.smithi170.stdout: python3-singledispatch noarch3.4.0.3-18.el8 @copr:copr.fedorainfracloud.org:ktdreyer:ceph-el8 43 k 2020-03-19T21:19:34.771 INFO:teuthology.orchestra.run.smithi170.stdout: python3-tempora noarch1.14.1-5.el8 @copr:copr.fedorainfracloud.org:ktdreyer:ceph-el8 82 k 2020-03-19T21:19:34.771 INFO:teuthology.orchestra.run.smithi170.stdout: python3-trustme noarch0.5.2-4.el8 @copr:copr.fedorainfracloud.org:ktdreyer:ceph-el8 51 k 2020-03-19T21:19:34.771 INFO:teuthology.orchestra.run.smithi170.stdout: python3-waitress noarch1.2.1-2.el8.1 @epel 1.0 M 2020-03-19T21:19:34.772 INFO:teuthology.orchestra.run.smithi170.stdout: python3-webencodings noarch0.5.1-6.el8 @rhel-8-for-x86_64-appstream-rpms 72 k 2020-03-19T21:19:34.772 INFO:teuthology.orchestra.run.smithi170.stdout: python3-webob noarch1.8.5-1.el8.1 @epel 1.2 M 2020-03-19T21:19:34.772 INFO:teuthology.orchestra.run.smithi170.stdout: python3-websocket-client noarch0.56.0-5.el8 @epel 180 k 2020-03-19T21:19:34.772 INFO:teuthology.orchestra.run.smithi170.stdout: python3-webtest noarch2.0.33-1.el8 @epel 289 k 2020-03-19T21:19:34.772 INFO:teuthology.orchestra.run.smithi170.stdout: python3-werkzeug noarch0.12.2-4.el8 @rhel-8-for-x86_64-appstream-rpms 2.0 M 2020-03-19T21:19:34.773 INFO:teuthology.orchestra.run.smithi170.stdout: python3-zc-lockfile noarch2.0-2.el8 @copr:copr.fedorainfracloud.org:ktdreyer:ceph-el8 35 k 2020-03-19T21:19:34.773 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:19:34.773 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction Summary 2020-03-19T21:19:34.773 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2020-03-19T21:19:34.773 INFO:teuthology.orchestra.run.smithi170.stdout:Remove 78 Packages 2020-03-19T21:19:34.773 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:19:34.774 INFO:teuthology.orchestra.run.smithi170.stdout:Freed space: 422 M 2020-03-19T21:19:34.774 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction check 2020-03-19T21:19:34.793 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction check succeeded. 2020-03-19T21:19:34.794 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction test 2020-03-19T21:19:34.882 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction test succeeded. 2020-03-19T21:19:34.882 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction 2020-03-19T21:19:35.797 INFO:teuthology.orchestra.run.smithi170.stdout: Preparing : 1/1 2020-03-19T21:19:35.823 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-mgr-rook-2:15.1.1-234.g787db24.el8.noarch 1/1 2020-03-19T21:19:35.824 INFO:teuthology.orchestra.run.smithi170.stdout: Erasing : ceph-mgr-rook-2:15.1.1-234.g787db24.el8.noarch 1/78 2020-03-19T21:19:35.966 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-mgr-rook-2:15.1.1-234.g787db24.el8.noarch 1/78 2020-03-19T21:19:35.984 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-mgr-2:15.1.1-234.g787db24.el8.x86_64 2/78 2020-03-19T21:19:35.984 INFO:teuthology.orchestra.run.smithi170.stdout: Erasing : ceph-mgr-2:15.1.1-234.g787db24.el8.x86_64 2/78 2020-03-19T21:19:36.238 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-mgr-2:15.1.1-234.g787db24.el8.x86_64 2/78 2020-03-19T21:19:36.444 INFO:teuthology.orchestra.run.smithi170.stdout: Erasing : ceph-mgr-modules-core-2:15.1.1-234.g787db24.el8.no 3/78 2020-03-19T21:19:36.735 INFO:teuthology.orchestra.run.smithi170.stdout: Erasing : python3-pecan-1.3.2-9.el8.noarch 4/78 2020-03-19T21:19:36.736 INFO:teuthology.orchestra.run.smithi170.stdout: Erasing : ceph-mgr-dashboard-2:15.1.1-234.g787db24.el8.noarc 5/78 2020-03-19T21:19:36.973 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-mgr-dashboard-2:15.1.1-234.g787db24.el8.noarc 5/78 2020-03-19T21:19:37.245 INFO:teuthology.orchestra.run.smithi170.stdout: Erasing : python3-cherrypy-18.4.0-1.el8.noarch 6/78 2020-03-19T21:19:37.478 INFO:teuthology.orchestra.run.smithi170.stdout: Erasing : python3-kubernetes-10.0.1-2.el8.noarch 7/78 2020-03-19T21:19:37.703 INFO:teuthology.orchestra.run.smithi170.stdout: Erasing : python3-google-auth-1:1.1.1-10.el8.noarch 8/78 2020-03-19T21:19:37.909 INFO:teuthology.orchestra.run.smithi170.stdout: Erasing : python3-cheroot-8.2.1-1.el8.noarch 9/78 2020-03-19T21:19:38.142 INFO:teuthology.orchestra.run.smithi170.stdout: Erasing : python3-webtest-2.0.33-1.el8.noarch 10/78 2020-03-19T21:19:38.142 INFO:teuthology.orchestra.run.smithi170.stdout: Erasing : ceph-mgr-diskprediction-local-2:15.1.1-234.g787db2 11/78 2020-03-19T21:19:38.396 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:15.1.1-234.g787db2 11/78 2020-03-19T21:19:38.624 INFO:teuthology.orchestra.run.smithi170.stdout: Erasing : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb623d 12/78 2020-03-19T21:19:38.646 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: rbd-mirror-2:15.1.1-234.g787db24.el8.x86_64 13/78 2020-03-19T21:19:38.646 INFO:teuthology.orchestra.run.smithi170.stdout: Erasing : rbd-mirror-2:15.1.1-234.g787db24.el8.x86_64 13/78 2020-03-19T21:19:38.822 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: rbd-mirror-2:15.1.1-234.g787db24.el8.x86_64 13/78 2020-03-19T21:19:38.822 INFO:teuthology.orchestra.run.smithi170.stdout: Erasing : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 14/78 2020-03-19T21:19:39.068 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 14/78 2020-03-19T21:19:39.276 INFO:teuthology.orchestra.run.smithi170.stdout: Erasing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 15/78 2020-03-19T21:19:39.410 INFO:teuthology.orchestra.run.smithi170.stdout: Erasing : python3-pyasn1-modules-0.3.7-6.el8.noarch 16/78 2020-03-19T21:19:39.600 INFO:teuthology.orchestra.run.smithi170.stdout: Erasing : python3-rsa-3.4.2-12.el8.noarch 17/78 2020-03-19T21:19:39.755 INFO:teuthology.orchestra.run.smithi170.stdout: Erasing : python3-requests-oauthlib-1.0.0-1.el8.noarch 18/78 2020-03-19T21:19:39.949 INFO:teuthology.orchestra.run.smithi170.stdout: Erasing : python3-oauthlib-2.1.0-1.el8.noarch 19/78 2020-03-19T21:19:40.100 INFO:teuthology.orchestra.run.smithi170.stdout: Erasing : python3-portend-2.6-1.el8.noarch 20/78 2020-03-19T21:19:40.274 INFO:teuthology.orchestra.run.smithi170.stdout: Erasing : python3-tempora-1.14.1-5.el8.noarch 21/78 2020-03-19T21:19:40.467 INFO:teuthology.orchestra.run.smithi170.stdout: Erasing : python3-jaraco-functools-2.0-4.el8.noarch 22/78 2020-03-19T21:19:40.620 INFO:teuthology.orchestra.run.smithi170.stdout: Erasing : python3-routes-2.4.1-12.el8.noarch 23/78 2020-03-19T21:19:40.824 INFO:teuthology.orchestra.run.smithi170.stdout: Erasing : python3-mako-1.0.6-13.el8.noarch 24/78 2020-03-19T21:19:40.824 INFO:teuthology.orchestra.run.smithi170.stdout: Erasing : ceph-mgr-cephadm-2:15.1.1-234.g787db24.el8.noarch 25/78 2020-03-19T21:19:40.977 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-mgr-cephadm-2:15.1.1-234.g787db24.el8.noarch 25/78 2020-03-19T21:19:41.188 INFO:teuthology.orchestra.run.smithi170.stdout: Erasing : python3-remoto-1.1.4-4.el8.noarch 26/78 2020-03-19T21:19:41.408 INFO:teuthology.orchestra.run.smithi170.stdout: Erasing : python3-execnet-1.7.1-1.el8.2.noarch 27/78 2020-03-19T21:19:41.541 INFO:teuthology.orchestra.run.smithi170.stdout: Erasing : python3-jsonpatch-1.21-2.el8.noarch 28/78 2020-03-19T21:19:41.699 INFO:teuthology.orchestra.run.smithi170.stdout: Erasing : python3-jsonpointer-1.10-11.el8.noarch 29/78 2020-03-19T21:19:41.866 INFO:teuthology.orchestra.run.smithi170.stdout: Erasing : python3-apipkg-1.5-6.el8.noarch 30/78 2020-03-19T21:19:42.040 INFO:teuthology.orchestra.run.smithi170.stdout: Erasing : python3-repoze-lru-0.7-6.el8.noarch 31/78 2020-03-19T21:19:42.208 INFO:teuthology.orchestra.run.smithi170.stdout: Erasing : python3-jaraco-6.2-6.el8.noarch 32/78 2020-03-19T21:19:42.359 INFO:teuthology.orchestra.run.smithi170.stdout: Erasing : python3-pytz-2017.2-9.el8.noarch 33/78 2020-03-19T21:19:42.546 INFO:teuthology.orchestra.run.smithi170.stdout: Erasing : python3-jwt-1.6.1-2.el8.noarch 34/78 2020-03-19T21:19:42.713 INFO:teuthology.orchestra.run.smithi170.stdout: Erasing : python3-pyasn1-0.3.7-6.el8.noarch 35/78 2020-03-19T21:19:42.937 INFO:teuthology.orchestra.run.smithi170.stdout: Erasing : python3-waitress-1.2.1-2.el8.1.noarch 36/78 2020-03-19T21:19:43.140 INFO:teuthology.orchestra.run.smithi170.stdout: Erasing : python3-webob-1.8.5-1.el8.1.noarch 37/78 2020-03-19T21:19:43.274 INFO:teuthology.orchestra.run.smithi170.stdout: Erasing : python3-more-itertools-7.2.0-3.el8.noarch 38/78 2020-03-19T21:19:43.441 INFO:teuthology.orchestra.run.smithi170.stdout: Erasing : python3-trustme-0.5.2-4.el8.noarch 39/78 2020-03-19T21:19:43.590 INFO:teuthology.orchestra.run.smithi170.stdout: Erasing : python3-cachetools-3.1.1-4.el8.noarch 40/78 2020-03-19T21:19:43.759 INFO:teuthology.orchestra.run.smithi170.stdout: Erasing : python3-certifi-2018.10.15-7.el8.noarch 41/78 2020-03-19T21:19:43.957 INFO:teuthology.orchestra.run.smithi170.stdout: Erasing : python3-websocket-client-0.56.0-5.el8.noarch 42/78 2020-03-19T21:19:44.090 INFO:teuthology.orchestra.run.smithi170.stdout: Erasing : python3-zc-lockfile-2.0-2.el8.noarch 43/78 2020-03-19T21:19:44.272 INFO:teuthology.orchestra.run.smithi170.stdout: Erasing : ceph-grafana-dashboards-2:15.1.1-234.g787db24.el8. 44/78 2020-03-19T21:19:44.442 INFO:teuthology.orchestra.run.smithi170.stdout: Erasing : python3-werkzeug-0.12.2-4.el8.noarch 45/78 2020-03-19T21:19:44.631 INFO:teuthology.orchestra.run.smithi170.stdout: Erasing : python3-logutils-0.3.5-11.el8.noarch 46/78 2020-03-19T21:19:44.758 INFO:teuthology.orchestra.run.smithi170.stdout: Erasing : python3-simplegeneric-0.8.1-17.el8.noarch 47/78 2020-03-19T21:19:44.922 INFO:teuthology.orchestra.run.smithi170.stdout: Erasing : python3-singledispatch-3.4.0.3-18.el8.noarch 48/78 2020-03-19T21:19:45.140 INFO:teuthology.orchestra.run.smithi170.stdout: Erasing : python3-lxml-4.2.3-1.el8.x86_64 49/78 2020-03-19T21:19:45.185 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-base-2:15.1.1-234.g787db24.el8.x86_64 50/78 2020-03-19T21:19:45.186 INFO:teuthology.orchestra.run.smithi170.stdout: Erasing : ceph-base-2:15.1.1-234.g787db24.el8.x86_64 50/78 2020-03-19T21:19:45.186 INFO:teuthology.orchestra.run.smithi170.stdout:warning: file /etc/logrotate.d/ceph: remove failed: No such file or directory 2020-03-19T21:19:45.186 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:19:45.526 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-base-2:15.1.1-234.g787db24.el8.x86_64 50/78 2020-03-19T21:19:45.526 INFO:teuthology.orchestra.run.smithi170.stdout: Erasing : ceph-common-2:15.1.1-234.g787db24.el8.x86_64 51/78 2020-03-19T21:19:45.526 INFO:teuthology.orchestra.run.smithi170.stdout:warning: directory /var/lib/ceph: remove failed: Device or resource busy 2020-03-19T21:19:45.526 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:19:45.772 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-common-2:15.1.1-234.g787db24.el8.x86_64 51/78 2020-03-19T21:19:45.958 INFO:teuthology.orchestra.run.smithi170.stdout: Erasing : python3-rgw-2:15.1.1-234.g787db24.el8.x86_64 52/78 2020-03-19T21:19:45.959 INFO:teuthology.orchestra.run.smithi170.stdout: Erasing : librgw2-2:15.1.1-234.g787db24.el8.x86_64 53/78 2020-03-19T21:19:46.165 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: librgw2-2:15.1.1-234.g787db24.el8.x86_64 53/78 2020-03-19T21:19:46.391 INFO:teuthology.orchestra.run.smithi170.stdout: Erasing : python3-cephfs-2:15.1.1-234.g787db24.el8.x86_64 54/78 2020-03-19T21:19:46.611 INFO:teuthology.orchestra.run.smithi170.stdout: Erasing : python3-numpy-1:1.14.3-9.el8.x86_64 55/78 2020-03-19T21:19:46.803 INFO:teuthology.orchestra.run.smithi170.stdout: Erasing : python3-html5lib-1:0.999999999-6.el8.noarch 56/78 2020-03-19T21:19:46.948 INFO:teuthology.orchestra.run.smithi170.stdout: Erasing : python3-webencodings-0.5.1-6.el8.noarch 57/78 2020-03-19T21:19:47.085 INFO:teuthology.orchestra.run.smithi170.stdout: Erasing : python3-ceph-argparse-2:15.1.1-234.g787db24.el8.x8 58/78 2020-03-19T21:19:47.223 INFO:teuthology.orchestra.run.smithi170.stdout: Erasing : python3-ceph-common-2:15.1.1-234.g787db24.el8.x86_ 59/78 2020-03-19T21:19:47.389 INFO:teuthology.orchestra.run.smithi170.stdout: Erasing : python3-prettytable-0.7.2-14.el8.noarch 60/78 2020-03-19T21:19:47.389 INFO:teuthology.orchestra.run.smithi170.stdout: Erasing : ceph-selinux-2:15.1.1-234.g787db24.el8.x86_64 61/78 2020-03-19T21:20:02.220 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-selinux-2:15.1.1-234.g787db24.el8.x86_64 61/78 2020-03-19T21:20:02.220 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /sys 2020-03-19T21:20:02.220 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /proc 2020-03-19T21:20:02.220 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /dev 2020-03-19T21:20:02.220 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /run 2020-03-19T21:20:02.220 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /mnt 2020-03-19T21:20:02.220 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /var/tmp 2020-03-19T21:20:02.221 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /home 2020-03-19T21:20:02.221 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /tmp 2020-03-19T21:20:02.221 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /dev 2020-03-19T21:20:02.221 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:20:02.481 INFO:teuthology.orchestra.run.smithi170.stdout: Erasing : python3-cssselect-0.9.2-10.el8.noarch 62/78 2020-03-19T21:20:02.481 INFO:teuthology.orchestra.run.smithi170.stdout: Erasing : openblas-0.3.3-2.el8.x86_64 63/78 2020-03-19T21:20:02.706 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: openblas-0.3.3-2.el8.x86_64 63/78 2020-03-19T21:20:02.706 INFO:teuthology.orchestra.run.smithi170.stdout: Erasing : openblas-threads-0.3.3-2.el8.x86_64 64/78 2020-03-19T21:20:02.895 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: openblas-threads-0.3.3-2.el8.x86_64 64/78 2020-03-19T21:20:02.895 INFO:teuthology.orchestra.run.smithi170.stdout: Erasing : libgfortran-8.3.1-4.5.el8.x86_64 65/78 2020-03-19T21:20:03.087 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: libgfortran-8.3.1-4.5.el8.x86_64 65/78 2020-03-19T21:20:03.245 INFO:teuthology.orchestra.run.smithi170.stdout: Erasing : leveldb-1.21-2.el8.x86_64 66/78 2020-03-19T21:20:03.386 INFO:teuthology.orchestra.run.smithi170.stdout: Erasing : gperftools-libs-2.7-6.el8.x86_64 67/78 2020-03-19T21:20:03.545 INFO:teuthology.orchestra.run.smithi170.stdout: Erasing : python3-rbd-2:15.1.1-234.g787db24.el8.x86_64 68/78 2020-03-19T21:20:03.720 INFO:teuthology.orchestra.run.smithi170.stdout: Erasing : python3-rados-2:15.1.1-234.g787db24.el8.x86_64 69/78 2020-03-19T21:20:03.878 INFO:teuthology.orchestra.run.smithi170.stdout: Erasing : libunwind-1.3.1-3.el8.x86_64 70/78 2020-03-19T21:20:03.887 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: libquadmath-8.3.1-4.5.el8.x86_64 71/78 2020-03-19T21:20:03.887 INFO:teuthology.orchestra.run.smithi170.stdout: Erasing : libquadmath-8.3.1-4.5.el8.x86_64 71/78 2020-03-19T21:20:04.137 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: libquadmath-8.3.1-4.5.el8.x86_64 71/78 2020-03-19T21:20:04.278 INFO:teuthology.orchestra.run.smithi170.stdout: Erasing : liboath-2.6.2-3.el8.x86_64 72/78 2020-03-19T21:20:04.395 INFO:teuthology.orchestra.run.smithi170.stdout: Erasing : librabbitmq-0.9.0-1.el8.x86_64 73/78 2020-03-19T21:20:04.395 INFO:teuthology.orchestra.run.smithi170.stdout: Erasing : librdkafka-0.11.4-1.el8.x86_64 74/78 2020-03-19T21:20:04.612 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: librdkafka-0.11.4-1.el8.x86_64 74/78 2020-03-19T21:20:04.612 INFO:teuthology.orchestra.run.smithi170.stdout: Erasing : libradosstriper1-2:15.1.1-234.g787db24.el8.x86_64 75/78 2020-03-19T21:20:04.787 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: libradosstriper1-2:15.1.1-234.g787db24.el8.x86_64 75/78 2020-03-19T21:20:04.954 INFO:teuthology.orchestra.run.smithi170.stdout: Erasing : libxslt-1.1.32-3.el8.x86_64 76/78 2020-03-19T21:20:05.146 INFO:teuthology.orchestra.run.smithi170.stdout: Erasing : python3-markupsafe-0.23-19.el8.x86_64 77/78 2020-03-19T21:20:05.338 INFO:teuthology.orchestra.run.smithi170.stdout: Erasing : python3-bcrypt-3.1.6-2.el8.1.x86_64 78/78 2020-03-19T21:20:05.624 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: python3-bcrypt-3.1.6-2.el8.1.x86_64 78/78 2020-03-19T21:20:05.624 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-base-2:15.1.1-234.g787db24.el8.x86_64 1/78 2020-03-19T21:20:05.624 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-common-2:15.1.1-234.g787db24.el8.x86_64 2/78 2020-03-19T21:20:05.624 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-grafana-dashboards-2:15.1.1-234.g787db24.el8. 3/78 2020-03-19T21:20:05.624 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mgr-2:15.1.1-234.g787db24.el8.x86_64 4/78 2020-03-19T21:20:05.624 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mgr-cephadm-2:15.1.1-234.g787db24.el8.noarch 5/78 2020-03-19T21:20:05.625 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mgr-dashboard-2:15.1.1-234.g787db24.el8.noarc 6/78 2020-03-19T21:20:05.625 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mgr-diskprediction-local-2:15.1.1-234.g787db2 7/78 2020-03-19T21:20:05.625 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mgr-modules-core-2:15.1.1-234.g787db24.el8.no 8/78 2020-03-19T21:20:05.625 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mgr-rook-2:15.1.1-234.g787db24.el8.noarch 9/78 2020-03-19T21:20:05.625 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-selinux-2:15.1.1-234.g787db24.el8.x86_64 10/78 2020-03-19T21:20:05.625 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : gperftools-libs-2.7-6.el8.x86_64 11/78 2020-03-19T21:20:05.625 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : leveldb-1.21-2.el8.x86_64 12/78 2020-03-19T21:20:05.626 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libgfortran-8.3.1-4.5.el8.x86_64 13/78 2020-03-19T21:20:05.626 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 14/78 2020-03-19T21:20:05.626 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libquadmath-8.3.1-4.5.el8.x86_64 15/78 2020-03-19T21:20:05.626 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librabbitmq-0.9.0-1.el8.x86_64 16/78 2020-03-19T21:20:05.626 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libradosstriper1-2:15.1.1-234.g787db24.el8.x86_64 17/78 2020-03-19T21:20:05.626 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librdkafka-0.11.4-1.el8.x86_64 18/78 2020-03-19T21:20:05.626 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librgw2-2:15.1.1-234.g787db24.el8.x86_64 19/78 2020-03-19T21:20:05.626 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 20/78 2020-03-19T21:20:05.627 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libxslt-1.1.32-3.el8.x86_64 21/78 2020-03-19T21:20:05.627 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : openblas-0.3.3-2.el8.x86_64 22/78 2020-03-19T21:20:05.627 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : openblas-threads-0.3.3-2.el8.x86_64 23/78 2020-03-19T21:20:05.627 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-apipkg-1.5-6.el8.noarch 24/78 2020-03-19T21:20:05.627 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 25/78 2020-03-19T21:20:05.627 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 26/78 2020-03-19T21:20:05.627 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 27/78 2020-03-19T21:20:05.628 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-ceph-argparse-2:15.1.1-234.g787db24.el8.x8 28/78 2020-03-19T21:20:05.632 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-ceph-common-2:15.1.1-234.g787db24.el8.x86_ 29/78 2020-03-19T21:20:05.632 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-cephfs-2:15.1.1-234.g787db24.el8.x86_64 30/78 2020-03-19T21:20:05.632 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 31/78 2020-03-19T21:20:05.633 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-cheroot-8.2.1-1.el8.noarch 32/78 2020-03-19T21:20:05.633 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 33/78 2020-03-19T21:20:05.633 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-cssselect-0.9.2-10.el8.noarch 34/78 2020-03-19T21:20:05.633 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-execnet-1.7.1-1.el8.2.noarch 35/78 2020-03-19T21:20:05.633 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 36/78 2020-03-19T21:20:05.633 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-html5lib-1:0.999999999-6.el8.noarch 37/78 2020-03-19T21:20:05.633 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 38/78 2020-03-19T21:20:05.634 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 39/78 2020-03-19T21:20:05.634 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 40/78 2020-03-19T21:20:05.634 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 41/78 2020-03-19T21:20:05.634 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 42/78 2020-03-19T21:20:05.634 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-kubernetes-10.0.1-2.el8.noarch 43/78 2020-03-19T21:20:05.634 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 44/78 2020-03-19T21:20:05.634 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-lxml-4.2.3-1.el8.x86_64 45/78 2020-03-19T21:20:05.635 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 46/78 2020-03-19T21:20:05.635 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 47/78 2020-03-19T21:20:05.635 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 48/78 2020-03-19T21:20:05.635 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-numpy-1:1.14.3-9.el8.x86_64 49/78 2020-03-19T21:20:05.635 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 50/78 2020-03-19T21:20:05.635 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 51/78 2020-03-19T21:20:05.636 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 52/78 2020-03-19T21:20:05.636 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-portend-2.6-1.el8.noarch 53/78 2020-03-19T21:20:05.636 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 54/78 2020-03-19T21:20:05.636 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 55/78 2020-03-19T21:20:05.636 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 56/78 2020-03-19T21:20:05.636 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 57/78 2020-03-19T21:20:05.636 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-rados-2:15.1.1-234.g787db24.el8.x86_64 58/78 2020-03-19T21:20:05.637 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-rbd-2:15.1.1-234.g787db24.el8.x86_64 59/78 2020-03-19T21:20:05.637 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-remoto-1.1.4-4.el8.noarch 60/78 2020-03-19T21:20:05.637 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 61/78 2020-03-19T21:20:05.637 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 62/78 2020-03-19T21:20:05.637 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-rgw-2:15.1.1-234.g787db24.el8.x86_64 63/78 2020-03-19T21:20:05.637 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 64/78 2020-03-19T21:20:05.638 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-rsa-3.4.2-12.el8.noarch 65/78 2020-03-19T21:20:05.638 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb623d 66/78 2020-03-19T21:20:05.638 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 67/78 2020-03-19T21:20:05.638 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 68/78 2020-03-19T21:20:05.638 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 69/78 2020-03-19T21:20:05.638 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-trustme-0.5.2-4.el8.noarch 70/78 2020-03-19T21:20:05.638 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-waitress-1.2.1-2.el8.1.noarch 71/78 2020-03-19T21:20:05.638 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-webencodings-0.5.1-6.el8.noarch 72/78 2020-03-19T21:20:05.639 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 73/78 2020-03-19T21:20:05.639 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 74/78 2020-03-19T21:20:05.639 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 75/78 2020-03-19T21:20:05.639 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 76/78 2020-03-19T21:20:05.639 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 77/78 2020-03-19T21:20:08.942 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : rbd-mirror-2:15.1.1-234.g787db24.el8.x86_64 78/78 2020-03-19T21:20:08.943 INFO:teuthology.orchestra.run.smithi170.stdout:Installed products updated. 2020-03-19T21:20:09.148 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:20:09.148 INFO:teuthology.orchestra.run.smithi170.stdout:Removed: 2020-03-19T21:20:09.148 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-base-2:15.1.1-234.g787db24.el8.x86_64 2020-03-19T21:20:09.148 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-rook-2:15.1.1-234.g787db24.el8.noarch 2020-03-19T21:20:09.148 INFO:teuthology.orchestra.run.smithi170.stdout: rbd-mirror-2:15.1.1-234.g787db24.el8.x86_64 2020-03-19T21:20:09.148 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-common-2:15.1.1-234.g787db24.el8.x86_64 2020-03-19T21:20:09.149 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-grafana-dashboards-2:15.1.1-234.g787db24.el8.noarch 2020-03-19T21:20:09.149 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-2:15.1.1-234.g787db24.el8.x86_64 2020-03-19T21:20:09.149 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-cephadm-2:15.1.1-234.g787db24.el8.noarch 2020-03-19T21:20:09.149 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-dashboard-2:15.1.1-234.g787db24.el8.noarch 2020-03-19T21:20:09.149 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-diskprediction-local-2:15.1.1-234.g787db24.el8.noarch 2020-03-19T21:20:09.149 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-modules-core-2:15.1.1-234.g787db24.el8.noarch 2020-03-19T21:20:09.149 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-selinux-2:15.1.1-234.g787db24.el8.x86_64 2020-03-19T21:20:09.150 INFO:teuthology.orchestra.run.smithi170.stdout: gperftools-libs-2.7-6.el8.x86_64 2020-03-19T21:20:09.150 INFO:teuthology.orchestra.run.smithi170.stdout: leveldb-1.21-2.el8.x86_64 2020-03-19T21:20:09.150 INFO:teuthology.orchestra.run.smithi170.stdout: libgfortran-8.3.1-4.5.el8.x86_64 2020-03-19T21:20:09.150 INFO:teuthology.orchestra.run.smithi170.stdout: liboath-2.6.2-3.el8.x86_64 2020-03-19T21:20:09.150 INFO:teuthology.orchestra.run.smithi170.stdout: libquadmath-8.3.1-4.5.el8.x86_64 2020-03-19T21:20:09.150 INFO:teuthology.orchestra.run.smithi170.stdout: librabbitmq-0.9.0-1.el8.x86_64 2020-03-19T21:20:09.150 INFO:teuthology.orchestra.run.smithi170.stdout: libradosstriper1-2:15.1.1-234.g787db24.el8.x86_64 2020-03-19T21:20:09.151 INFO:teuthology.orchestra.run.smithi170.stdout: librdkafka-0.11.4-1.el8.x86_64 2020-03-19T21:20:09.151 INFO:teuthology.orchestra.run.smithi170.stdout: librgw2-2:15.1.1-234.g787db24.el8.x86_64 2020-03-19T21:20:09.151 INFO:teuthology.orchestra.run.smithi170.stdout: libunwind-1.3.1-3.el8.x86_64 2020-03-19T21:20:09.151 INFO:teuthology.orchestra.run.smithi170.stdout: libxslt-1.1.32-3.el8.x86_64 2020-03-19T21:20:09.151 INFO:teuthology.orchestra.run.smithi170.stdout: openblas-0.3.3-2.el8.x86_64 2020-03-19T21:20:09.151 INFO:teuthology.orchestra.run.smithi170.stdout: openblas-threads-0.3.3-2.el8.x86_64 2020-03-19T21:20:09.151 INFO:teuthology.orchestra.run.smithi170.stdout: python3-apipkg-1.5-6.el8.noarch 2020-03-19T21:20:09.152 INFO:teuthology.orchestra.run.smithi170.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2020-03-19T21:20:09.152 INFO:teuthology.orchestra.run.smithi170.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2020-03-19T21:20:09.152 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cachetools-3.1.1-4.el8.noarch 2020-03-19T21:20:09.152 INFO:teuthology.orchestra.run.smithi170.stdout: python3-ceph-argparse-2:15.1.1-234.g787db24.el8.x86_64 2020-03-19T21:20:09.152 INFO:teuthology.orchestra.run.smithi170.stdout: python3-ceph-common-2:15.1.1-234.g787db24.el8.x86_64 2020-03-19T21:20:09.152 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cephfs-2:15.1.1-234.g787db24.el8.x86_64 2020-03-19T21:20:09.152 INFO:teuthology.orchestra.run.smithi170.stdout: python3-certifi-2018.10.15-7.el8.noarch 2020-03-19T21:20:09.152 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cheroot-8.2.1-1.el8.noarch 2020-03-19T21:20:09.153 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2020-03-19T21:20:09.153 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cssselect-0.9.2-10.el8.noarch 2020-03-19T21:20:09.153 INFO:teuthology.orchestra.run.smithi170.stdout: python3-execnet-1.7.1-1.el8.2.noarch 2020-03-19T21:20:09.153 INFO:teuthology.orchestra.run.smithi170.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2020-03-19T21:20:09.153 INFO:teuthology.orchestra.run.smithi170.stdout: python3-html5lib-1:0.999999999-6.el8.noarch 2020-03-19T21:20:09.153 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco-6.2-6.el8.noarch 2020-03-19T21:20:09.154 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2020-03-19T21:20:09.154 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jsonpatch-1.21-2.el8.noarch 2020-03-19T21:20:09.154 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jsonpointer-1.10-11.el8.noarch 2020-03-19T21:20:09.155 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jwt-1.6.1-2.el8.noarch 2020-03-19T21:20:09.155 INFO:teuthology.orchestra.run.smithi170.stdout: python3-kubernetes-10.0.1-2.el8.noarch 2020-03-19T21:20:09.155 INFO:teuthology.orchestra.run.smithi170.stdout: python3-logutils-0.3.5-11.el8.noarch 2020-03-19T21:20:09.155 INFO:teuthology.orchestra.run.smithi170.stdout: python3-lxml-4.2.3-1.el8.x86_64 2020-03-19T21:20:09.155 INFO:teuthology.orchestra.run.smithi170.stdout: python3-mako-1.0.6-13.el8.noarch 2020-03-19T21:20:09.155 INFO:teuthology.orchestra.run.smithi170.stdout: python3-markupsafe-0.23-19.el8.x86_64 2020-03-19T21:20:09.156 INFO:teuthology.orchestra.run.smithi170.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2020-03-19T21:20:09.156 INFO:teuthology.orchestra.run.smithi170.stdout: python3-numpy-1:1.14.3-9.el8.x86_64 2020-03-19T21:20:09.156 INFO:teuthology.orchestra.run.smithi170.stdout: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 2020-03-19T21:20:09.169 INFO:teuthology.orchestra.run.smithi170.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2020-03-19T21:20:09.170 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pecan-1.3.2-9.el8.noarch 2020-03-19T21:20:09.170 INFO:teuthology.orchestra.run.smithi170.stdout: python3-portend-2.6-1.el8.noarch 2020-03-19T21:20:09.170 INFO:teuthology.orchestra.run.smithi170.stdout: python3-prettytable-0.7.2-14.el8.noarch 2020-03-19T21:20:09.170 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2020-03-19T21:20:09.170 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2020-03-19T21:20:09.171 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pytz-2017.2-9.el8.noarch 2020-03-19T21:20:09.171 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rados-2:15.1.1-234.g787db24.el8.x86_64 2020-03-19T21:20:09.171 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rbd-2:15.1.1-234.g787db24.el8.x86_64 2020-03-19T21:20:09.171 INFO:teuthology.orchestra.run.smithi170.stdout: python3-remoto-1.1.4-4.el8.noarch 2020-03-19T21:20:09.171 INFO:teuthology.orchestra.run.smithi170.stdout: python3-repoze-lru-0.7-6.el8.noarch 2020-03-19T21:20:09.171 INFO:teuthology.orchestra.run.smithi170.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2020-03-19T21:20:09.171 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rgw-2:15.1.1-234.g787db24.el8.x86_64 2020-03-19T21:20:09.171 INFO:teuthology.orchestra.run.smithi170.stdout: python3-routes-2.4.1-12.el8.noarch 2020-03-19T21:20:09.172 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rsa-3.4.2-12.el8.noarch 2020-03-19T21:20:09.172 INFO:teuthology.orchestra.run.smithi170.stdout: python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb623d7.x86_64 2020-03-19T21:20:09.172 INFO:teuthology.orchestra.run.smithi170.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2020-03-19T21:20:09.172 INFO:teuthology.orchestra.run.smithi170.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2020-03-19T21:20:09.172 INFO:teuthology.orchestra.run.smithi170.stdout: python3-tempora-1.14.1-5.el8.noarch 2020-03-19T21:20:09.172 INFO:teuthology.orchestra.run.smithi170.stdout: python3-trustme-0.5.2-4.el8.noarch 2020-03-19T21:20:09.172 INFO:teuthology.orchestra.run.smithi170.stdout: python3-waitress-1.2.1-2.el8.1.noarch 2020-03-19T21:20:09.173 INFO:teuthology.orchestra.run.smithi170.stdout: python3-webencodings-0.5.1-6.el8.noarch 2020-03-19T21:20:09.173 INFO:teuthology.orchestra.run.smithi170.stdout: python3-webob-1.8.5-1.el8.1.noarch 2020-03-19T21:20:09.173 INFO:teuthology.orchestra.run.smithi170.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2020-03-19T21:20:09.173 INFO:teuthology.orchestra.run.smithi170.stdout: python3-webtest-2.0.33-1.el8.noarch 2020-03-19T21:20:09.173 INFO:teuthology.orchestra.run.smithi170.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2020-03-19T21:20:09.173 INFO:teuthology.orchestra.run.smithi170.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2020-03-19T21:20:09.174 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:20:09.174 INFO:teuthology.orchestra.run.smithi170.stdout:Complete! 2020-03-19T21:20:09.666 INFO:teuthology.orchestra.run.smithi170.stdout:Updating Subscription Management repositories. 2020-03-19T21:20:10.752 INFO:teuthology.orchestra.run.smithi170.stderr:Modular dependency problem: 2020-03-19T21:20:10.752 INFO:teuthology.orchestra.run.smithi170.stderr: 2020-03-19T21:20:10.752 INFO:teuthology.orchestra.run.smithi170.stderr: Problem: conflicting requests 2020-03-19T21:20:10.752 INFO:teuthology.orchestra.run.smithi170.stderr: - nothing provides module(perl:5.26) needed by module perl-YAML:1.24:8010020190322131811:a5949e2e-0.x86_64 2020-03-19T21:20:10.786 INFO:teuthology.orchestra.run.smithi170.stdout:Dependencies resolved. 2020-03-19T21:20:10.786 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2020-03-19T21:20:10.786 INFO:teuthology.orchestra.run.smithi170.stdout: Package Architecture Version Repository Size 2020-03-19T21:20:10.786 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2020-03-19T21:20:10.787 INFO:teuthology.orchestra.run.smithi170.stdout:Removing: 2020-03-19T21:20:10.787 INFO:teuthology.orchestra.run.smithi170.stdout: cephadm x86_64 2:15.1.1-234.g787db24.el8 @ceph 148 k 2020-03-19T21:20:10.787 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:20:10.787 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction Summary 2020-03-19T21:20:10.788 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2020-03-19T21:20:10.788 INFO:teuthology.orchestra.run.smithi170.stdout:Remove 1 Package 2020-03-19T21:20:10.788 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:20:10.788 INFO:teuthology.orchestra.run.smithi170.stdout:Freed space: 148 k 2020-03-19T21:20:10.789 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction check 2020-03-19T21:20:10.790 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction check succeeded. 2020-03-19T21:20:10.790 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction test 2020-03-19T21:20:10.790 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction test succeeded. 2020-03-19T21:20:10.790 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction 2020-03-19T21:20:10.825 INFO:teuthology.orchestra.run.smithi170.stdout: Preparing : 1/1 2020-03-19T21:20:10.826 INFO:teuthology.orchestra.run.smithi170.stdout: Erasing : cephadm-2:15.1.1-234.g787db24.el8.x86_64 1/1 2020-03-19T21:20:11.806 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: cephadm-2:15.1.1-234.g787db24.el8.x86_64 1/1 2020-03-19T21:20:11.806 INFO:teuthology.orchestra.run.smithi170.stdout:userdel: cephadm mail spool (/var/spool/mail/cephadm) not found 2020-03-19T21:20:11.806 INFO:teuthology.orchestra.run.smithi170.stdout:userdel: cephadm home directory (/var/lib/cephadm) not found 2020-03-19T21:20:11.806 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:20:14.102 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : cephadm-2:15.1.1-234.g787db24.el8.x86_64 1/1 2020-03-19T21:20:14.102 INFO:teuthology.orchestra.run.smithi170.stdout:Installed products updated. 2020-03-19T21:20:14.291 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:20:14.291 INFO:teuthology.orchestra.run.smithi170.stdout:Removed: 2020-03-19T21:20:14.292 INFO:teuthology.orchestra.run.smithi170.stdout: cephadm-2:15.1.1-234.g787db24.el8.x86_64 2020-03-19T21:20:14.292 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:20:14.292 INFO:teuthology.orchestra.run.smithi170.stdout:Complete! 2020-03-19T21:20:14.803 INFO:teuthology.orchestra.run.smithi170.stdout:Updating Subscription Management repositories. 2020-03-19T21:20:15.345 INFO:teuthology.orchestra.run.smithi170.stderr:Modular dependency problem: 2020-03-19T21:20:15.345 INFO:teuthology.orchestra.run.smithi170.stderr: 2020-03-19T21:20:15.345 INFO:teuthology.orchestra.run.smithi170.stderr: Problem: conflicting requests 2020-03-19T21:20:15.345 INFO:teuthology.orchestra.run.smithi170.stderr: - nothing provides module(perl:5.26) needed by module perl-YAML:1.24:8010020190322131811:a5949e2e-0.x86_64 2020-03-19T21:20:15.375 INFO:teuthology.orchestra.run.smithi170.stdout:Dependencies resolved. 2020-03-19T21:20:15.376 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2020-03-19T21:20:15.376 INFO:teuthology.orchestra.run.smithi170.stdout: Package Arch Version Repo Size 2020-03-19T21:20:15.377 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2020-03-19T21:20:15.377 INFO:teuthology.orchestra.run.smithi170.stdout:Removing: 2020-03-19T21:20:15.377 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-immutable-object-cache x86_64 2:15.1.1-234.g787db24.el8 @ceph 743 k 2020-03-19T21:20:15.377 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:20:15.377 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction Summary 2020-03-19T21:20:15.377 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2020-03-19T21:20:15.378 INFO:teuthology.orchestra.run.smithi170.stdout:Remove 1 Package 2020-03-19T21:20:15.378 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:20:15.378 INFO:teuthology.orchestra.run.smithi170.stdout:Freed space: 743 k 2020-03-19T21:20:15.378 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction check 2020-03-19T21:20:15.379 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction check succeeded. 2020-03-19T21:20:15.380 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction test 2020-03-19T21:20:15.406 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction test succeeded. 2020-03-19T21:20:15.406 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction 2020-03-19T21:20:15.466 INFO:teuthology.orchestra.run.smithi170.stdout: Preparing : 1/1 2020-03-19T21:20:15.503 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-immutable-object-cache-2:15.1.1-234.g787db24.el 1/1 2020-03-19T21:20:15.504 INFO:teuthology.orchestra.run.smithi170.stdout: Erasing : ceph-immutable-object-cache-2:15.1.1-234.g787db24.el 1/1 2020-03-19T21:20:15.920 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-immutable-object-cache-2:15.1.1-234.g787db24.el 1/1 2020-03-19T21:20:17.868 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-immutable-object-cache-2:15.1.1-234.g787db24.el 1/1 2020-03-19T21:20:17.868 INFO:teuthology.orchestra.run.smithi170.stdout:Installed products updated. 2020-03-19T21:20:18.049 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:20:18.049 INFO:teuthology.orchestra.run.smithi170.stdout:Removed: 2020-03-19T21:20:18.049 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-immutable-object-cache-2:15.1.1-234.g787db24.el8.x86_64 2020-03-19T21:20:18.050 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:20:18.050 INFO:teuthology.orchestra.run.smithi170.stdout:Complete! 2020-03-19T21:20:18.600 INFO:teuthology.orchestra.run.smithi170.stdout:Updating Subscription Management repositories. 2020-03-19T21:20:19.120 INFO:teuthology.orchestra.run.smithi170.stderr:Modular dependency problem: 2020-03-19T21:20:19.120 INFO:teuthology.orchestra.run.smithi170.stderr: 2020-03-19T21:20:19.120 INFO:teuthology.orchestra.run.smithi170.stderr: Problem: conflicting requests 2020-03-19T21:20:19.120 INFO:teuthology.orchestra.run.smithi170.stderr: - nothing provides module(perl:5.26) needed by module perl-YAML:1.24:8010020190322131811:a5949e2e-0.x86_64 2020-03-19T21:20:19.121 INFO:teuthology.orchestra.run.smithi170.stdout:No match for argument: ceph-mgr 2020-03-19T21:20:19.121 INFO:teuthology.orchestra.run.smithi170.stderr:No packages marked for removal. 2020-03-19T21:20:19.125 INFO:teuthology.orchestra.run.smithi170.stdout:Dependencies resolved. 2020-03-19T21:20:19.126 INFO:teuthology.orchestra.run.smithi170.stdout:Nothing to do. 2020-03-19T21:20:19.126 INFO:teuthology.orchestra.run.smithi170.stdout:Complete! 2020-03-19T21:20:19.593 INFO:teuthology.orchestra.run.smithi170.stdout:Updating Subscription Management repositories. 2020-03-19T21:20:20.090 INFO:teuthology.orchestra.run.smithi170.stderr:Modular dependency problem: 2020-03-19T21:20:20.090 INFO:teuthology.orchestra.run.smithi170.stderr: 2020-03-19T21:20:20.090 INFO:teuthology.orchestra.run.smithi170.stderr: Problem: conflicting requests 2020-03-19T21:20:20.090 INFO:teuthology.orchestra.run.smithi170.stderr: - nothing provides module(perl:5.26) needed by module perl-YAML:1.24:8010020190322131811:a5949e2e-0.x86_64 2020-03-19T21:20:20.091 INFO:teuthology.orchestra.run.smithi170.stdout:No match for argument: ceph-mgr-dashboard 2020-03-19T21:20:20.092 INFO:teuthology.orchestra.run.smithi170.stderr:No packages marked for removal. 2020-03-19T21:20:20.095 INFO:teuthology.orchestra.run.smithi170.stdout:Dependencies resolved. 2020-03-19T21:20:20.096 INFO:teuthology.orchestra.run.smithi170.stdout:Nothing to do. 2020-03-19T21:20:20.097 INFO:teuthology.orchestra.run.smithi170.stdout:Complete! 2020-03-19T21:20:20.578 INFO:teuthology.orchestra.run.smithi170.stdout:Updating Subscription Management repositories. 2020-03-19T21:20:21.152 INFO:teuthology.orchestra.run.smithi170.stderr:Modular dependency problem: 2020-03-19T21:20:21.152 INFO:teuthology.orchestra.run.smithi170.stderr: 2020-03-19T21:20:21.152 INFO:teuthology.orchestra.run.smithi170.stderr: Problem: conflicting requests 2020-03-19T21:20:21.152 INFO:teuthology.orchestra.run.smithi170.stderr: - nothing provides module(perl:5.26) needed by module perl-YAML:1.24:8010020190322131811:a5949e2e-0.x86_64 2020-03-19T21:20:21.153 INFO:teuthology.orchestra.run.smithi170.stdout:No match for argument: ceph-mgr-diskprediction-local 2020-03-19T21:20:21.153 INFO:teuthology.orchestra.run.smithi170.stderr:No packages marked for removal. 2020-03-19T21:20:21.158 INFO:teuthology.orchestra.run.smithi170.stdout:Dependencies resolved. 2020-03-19T21:20:21.159 INFO:teuthology.orchestra.run.smithi170.stdout:Nothing to do. 2020-03-19T21:20:21.159 INFO:teuthology.orchestra.run.smithi170.stdout:Complete! 2020-03-19T21:20:21.608 INFO:teuthology.orchestra.run.smithi170.stdout:Updating Subscription Management repositories. 2020-03-19T21:20:22.080 INFO:teuthology.orchestra.run.smithi170.stderr:Modular dependency problem: 2020-03-19T21:20:22.080 INFO:teuthology.orchestra.run.smithi170.stderr: 2020-03-19T21:20:22.080 INFO:teuthology.orchestra.run.smithi170.stderr: Problem: conflicting requests 2020-03-19T21:20:22.080 INFO:teuthology.orchestra.run.smithi170.stderr: - nothing provides module(perl:5.26) needed by module perl-YAML:1.24:8010020190322131811:a5949e2e-0.x86_64 2020-03-19T21:20:22.081 INFO:teuthology.orchestra.run.smithi170.stdout:No match for argument: ceph-mgr-rook 2020-03-19T21:20:22.081 INFO:teuthology.orchestra.run.smithi170.stderr:No packages marked for removal. 2020-03-19T21:20:22.086 INFO:teuthology.orchestra.run.smithi170.stdout:Dependencies resolved. 2020-03-19T21:20:22.087 INFO:teuthology.orchestra.run.smithi170.stdout:Nothing to do. 2020-03-19T21:20:22.087 INFO:teuthology.orchestra.run.smithi170.stdout:Complete! 2020-03-19T21:20:22.583 INFO:teuthology.orchestra.run.smithi170.stdout:Updating Subscription Management repositories. 2020-03-19T21:20:23.084 INFO:teuthology.orchestra.run.smithi170.stderr:Modular dependency problem: 2020-03-19T21:20:23.084 INFO:teuthology.orchestra.run.smithi170.stderr: 2020-03-19T21:20:23.084 INFO:teuthology.orchestra.run.smithi170.stderr: Problem: conflicting requests 2020-03-19T21:20:23.084 INFO:teuthology.orchestra.run.smithi170.stderr: - nothing provides module(perl:5.26) needed by module perl-YAML:1.24:8010020190322131811:a5949e2e-0.x86_64 2020-03-19T21:20:23.085 INFO:teuthology.orchestra.run.smithi170.stderr:No packages marked for removal. 2020-03-19T21:20:23.085 INFO:teuthology.orchestra.run.smithi170.stdout:No match for argument: ceph-mgr-cephadm 2020-03-19T21:20:23.088 INFO:teuthology.orchestra.run.smithi170.stdout:Dependencies resolved. 2020-03-19T21:20:23.089 INFO:teuthology.orchestra.run.smithi170.stdout:Nothing to do. 2020-03-19T21:20:23.089 INFO:teuthology.orchestra.run.smithi170.stdout:Complete! 2020-03-19T21:20:23.533 INFO:teuthology.orchestra.run.smithi170.stdout:Updating Subscription Management repositories. 2020-03-19T21:20:24.040 INFO:teuthology.orchestra.run.smithi170.stderr:Modular dependency problem: 2020-03-19T21:20:24.040 INFO:teuthology.orchestra.run.smithi170.stderr: 2020-03-19T21:20:24.041 INFO:teuthology.orchestra.run.smithi170.stderr: Problem: conflicting requests 2020-03-19T21:20:24.041 INFO:teuthology.orchestra.run.smithi170.stderr: - nothing provides module(perl:5.26) needed by module perl-YAML:1.24:8010020190322131811:a5949e2e-0.x86_64 2020-03-19T21:20:24.071 INFO:teuthology.orchestra.run.smithi170.stdout:Dependencies resolved. 2020-03-19T21:20:24.072 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2020-03-19T21:20:24.072 INFO:teuthology.orchestra.run.smithi170.stdout: Package Arch Version Repository Size 2020-03-19T21:20:24.072 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2020-03-19T21:20:24.072 INFO:teuthology.orchestra.run.smithi170.stdout:Removing: 2020-03-19T21:20:24.072 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-fuse x86_64 2:15.1.1-234.g787db24.el8 2020-03-19T21:20:24.073 INFO:teuthology.orchestra.run.smithi170.stdout: @ceph 1.9 M 2020-03-19T21:20:24.073 INFO:teuthology.orchestra.run.smithi170.stdout:Removing unused dependencies: 2020-03-19T21:20:24.073 INFO:teuthology.orchestra.run.smithi170.stdout: fuse x86_64 2.9.7-12.el8 @rhel-8-for-x86_64-baseos-rpms 206 k 2020-03-19T21:20:24.073 INFO:teuthology.orchestra.run.smithi170.stdout: fuse-common x86_64 3.2.1-12.el8 @rhel-8-for-x86_64-baseos-rpms 4.7 k 2020-03-19T21:20:24.073 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:20:24.073 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction Summary 2020-03-19T21:20:24.074 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2020-03-19T21:20:24.074 INFO:teuthology.orchestra.run.smithi170.stdout:Remove 3 Packages 2020-03-19T21:20:24.074 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:20:24.074 INFO:teuthology.orchestra.run.smithi170.stdout:Freed space: 2.1 M 2020-03-19T21:20:24.074 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction check 2020-03-19T21:20:24.076 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction check succeeded. 2020-03-19T21:20:24.076 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction test 2020-03-19T21:20:24.112 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction test succeeded. 2020-03-19T21:20:24.112 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction 2020-03-19T21:20:24.235 INFO:teuthology.orchestra.run.smithi170.stdout: Preparing : 1/1 2020-03-19T21:20:24.244 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-fuse-2:15.1.1-234.g787db24.el8.x86_64 1/1 2020-03-19T21:20:24.245 INFO:teuthology.orchestra.run.smithi170.stdout: Erasing : ceph-fuse-2:15.1.1-234.g787db24.el8.x86_64 1/3 2020-03-19T21:20:24.500 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-fuse-2:15.1.1-234.g787db24.el8.x86_64 1/3 2020-03-19T21:20:24.649 INFO:teuthology.orchestra.run.smithi170.stdout: Erasing : fuse-2.9.7-12.el8.x86_64 2/3 2020-03-19T21:20:24.649 INFO:teuthology.orchestra.run.smithi170.stdout: Erasing : fuse-common-3.2.1-12.el8.x86_64 3/3 2020-03-19T21:20:24.650 INFO:teuthology.orchestra.run.smithi170.stdout:warning: /etc/fuse.conf saved as /etc/fuse.conf.rpmsave 2020-03-19T21:20:24.650 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:20:25.011 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: fuse-common-3.2.1-12.el8.x86_64 3/3 2020-03-19T21:20:25.011 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-fuse-2:15.1.1-234.g787db24.el8.x86_64 1/3 2020-03-19T21:20:25.011 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : fuse-2.9.7-12.el8.x86_64 2/3 2020-03-19T21:20:27.127 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : fuse-common-3.2.1-12.el8.x86_64 3/3 2020-03-19T21:20:27.128 INFO:teuthology.orchestra.run.smithi170.stdout:Installed products updated. 2020-03-19T21:20:27.339 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:20:27.339 INFO:teuthology.orchestra.run.smithi170.stdout:Removed: 2020-03-19T21:20:27.339 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-fuse-2:15.1.1-234.g787db24.el8.x86_64 fuse-2.9.7-12.el8.x86_64 2020-03-19T21:20:27.339 INFO:teuthology.orchestra.run.smithi170.stdout: fuse-common-3.2.1-12.el8.x86_64 2020-03-19T21:20:27.339 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:20:27.340 INFO:teuthology.orchestra.run.smithi170.stdout:Complete! 2020-03-19T21:20:27.864 INFO:teuthology.orchestra.run.smithi170.stdout:Updating Subscription Management repositories. 2020-03-19T21:20:28.459 INFO:teuthology.orchestra.run.smithi170.stderr:Modular dependency problem: 2020-03-19T21:20:28.537 INFO:teuthology.orchestra.run.smithi170.stderr: 2020-03-19T21:20:28.538 INFO:teuthology.orchestra.run.smithi170.stderr: Problem: conflicting requests 2020-03-19T21:20:28.538 INFO:teuthology.orchestra.run.smithi170.stderr: - nothing provides module(perl:5.26) needed by module perl-YAML:1.24:8010020190322131811:a5949e2e-0.x86_64 2020-03-19T21:20:28.539 INFO:teuthology.orchestra.run.smithi170.stdout:Dependencies resolved. 2020-03-19T21:20:28.539 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2020-03-19T21:20:28.539 INFO:teuthology.orchestra.run.smithi170.stdout: Package Arch Version Repo Size 2020-03-19T21:20:28.540 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2020-03-19T21:20:28.540 INFO:teuthology.orchestra.run.smithi170.stdout:Removing: 2020-03-19T21:20:28.540 INFO:teuthology.orchestra.run.smithi170.stdout: librados-devel x86_64 2:15.1.1-234.g787db24.el8 @ceph 487 k 2020-03-19T21:20:28.540 INFO:teuthology.orchestra.run.smithi170.stdout:Removing dependent packages: 2020-03-19T21:20:28.540 INFO:teuthology.orchestra.run.smithi170.stdout: libcephfs-devel x86_64 2:15.1.1-234.g787db24.el8 @ceph 76 k 2020-03-19T21:20:28.540 INFO:teuthology.orchestra.run.smithi170.stdout:Removing unused dependencies: 2020-03-19T21:20:28.540 INFO:teuthology.orchestra.run.smithi170.stdout: libcephfs2 x86_64 2:15.1.1-234.g787db24.el8 @ceph 1.6 M 2020-03-19T21:20:28.541 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:20:28.541 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction Summary 2020-03-19T21:20:28.541 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2020-03-19T21:20:28.541 INFO:teuthology.orchestra.run.smithi170.stdout:Remove 3 Packages 2020-03-19T21:20:28.541 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:20:28.541 INFO:teuthology.orchestra.run.smithi170.stdout:Freed space: 2.2 M 2020-03-19T21:20:28.541 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction check 2020-03-19T21:20:28.541 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction check succeeded. 2020-03-19T21:20:28.542 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction test 2020-03-19T21:20:28.542 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction test succeeded. 2020-03-19T21:20:28.542 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction 2020-03-19T21:20:28.612 INFO:teuthology.orchestra.run.smithi170.stdout: Preparing : 1/1 2020-03-19T21:20:28.765 INFO:teuthology.orchestra.run.smithi170.stdout: Erasing : libcephfs-devel-2:15.1.1-234.g787db24.el8.x86_64 1/3 2020-03-19T21:20:28.765 INFO:teuthology.orchestra.run.smithi170.stdout: Erasing : libcephfs2-2:15.1.1-234.g787db24.el8.x86_64 2/3 2020-03-19T21:20:28.999 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: libcephfs2-2:15.1.1-234.g787db24.el8.x86_64 2/3 2020-03-19T21:20:29.126 INFO:teuthology.orchestra.run.smithi170.stdout: Erasing : librados-devel-2:15.1.1-234.g787db24.el8.x86_64 3/3 2020-03-19T21:20:29.328 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: librados-devel-2:15.1.1-234.g787db24.el8.x86_64 3/3 2020-03-19T21:20:29.328 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libcephfs-devel-2:15.1.1-234.g787db24.el8.x86_64 1/3 2020-03-19T21:20:29.328 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libcephfs2-2:15.1.1-234.g787db24.el8.x86_64 2/3 2020-03-19T21:20:31.409 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librados-devel-2:15.1.1-234.g787db24.el8.x86_64 3/3 2020-03-19T21:20:31.409 INFO:teuthology.orchestra.run.smithi170.stdout:Installed products updated. 2020-03-19T21:20:31.613 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:20:31.613 INFO:teuthology.orchestra.run.smithi170.stdout:Removed: 2020-03-19T21:20:31.614 INFO:teuthology.orchestra.run.smithi170.stdout: librados-devel-2:15.1.1-234.g787db24.el8.x86_64 2020-03-19T21:20:31.614 INFO:teuthology.orchestra.run.smithi170.stdout: libcephfs-devel-2:15.1.1-234.g787db24.el8.x86_64 2020-03-19T21:20:31.614 INFO:teuthology.orchestra.run.smithi170.stdout: libcephfs2-2:15.1.1-234.g787db24.el8.x86_64 2020-03-19T21:20:31.614 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:20:31.614 INFO:teuthology.orchestra.run.smithi170.stdout:Complete! 2020-03-19T21:20:32.150 INFO:teuthology.orchestra.run.smithi170.stdout:Updating Subscription Management repositories. 2020-03-19T21:20:32.644 INFO:teuthology.orchestra.run.smithi170.stderr:Modular dependency problem: 2020-03-19T21:20:32.645 INFO:teuthology.orchestra.run.smithi170.stderr: 2020-03-19T21:20:32.645 INFO:teuthology.orchestra.run.smithi170.stderr: Problem: conflicting requests 2020-03-19T21:20:32.645 INFO:teuthology.orchestra.run.smithi170.stderr: - nothing provides module(perl:5.26) needed by module perl-YAML:1.24:8010020190322131811:a5949e2e-0.x86_64 2020-03-19T21:20:32.646 INFO:teuthology.orchestra.run.smithi170.stdout:No match for argument: libcephfs2 2020-03-19T21:20:32.646 INFO:teuthology.orchestra.run.smithi170.stderr:No packages marked for removal. 2020-03-19T21:20:32.650 INFO:teuthology.orchestra.run.smithi170.stdout:Dependencies resolved. 2020-03-19T21:20:32.651 INFO:teuthology.orchestra.run.smithi170.stdout:Nothing to do. 2020-03-19T21:20:32.651 INFO:teuthology.orchestra.run.smithi170.stdout:Complete! 2020-03-19T21:20:33.113 INFO:teuthology.orchestra.run.smithi170.stdout:Updating Subscription Management repositories. 2020-03-19T21:20:33.634 INFO:teuthology.orchestra.run.smithi170.stderr:Modular dependency problem: 2020-03-19T21:20:33.634 INFO:teuthology.orchestra.run.smithi170.stderr: 2020-03-19T21:20:33.634 INFO:teuthology.orchestra.run.smithi170.stderr: Problem: conflicting requests 2020-03-19T21:20:33.634 INFO:teuthology.orchestra.run.smithi170.stderr: - nothing provides module(perl:5.26) needed by module perl-YAML:1.24:8010020190322131811:a5949e2e-0.x86_64 2020-03-19T21:20:33.635 INFO:teuthology.orchestra.run.smithi170.stdout:No match for argument: libcephfs-devel 2020-03-19T21:20:33.635 INFO:teuthology.orchestra.run.smithi170.stderr:No packages marked for removal. 2020-03-19T21:20:33.641 INFO:teuthology.orchestra.run.smithi170.stdout:Dependencies resolved. 2020-03-19T21:20:33.641 INFO:teuthology.orchestra.run.smithi170.stdout:Nothing to do. 2020-03-19T21:20:33.641 INFO:teuthology.orchestra.run.smithi170.stdout:Complete! 2020-03-19T21:20:34.107 INFO:teuthology.orchestra.run.smithi170.stdout:Updating Subscription Management repositories. 2020-03-19T21:20:34.628 INFO:teuthology.orchestra.run.smithi170.stderr:Modular dependency problem: 2020-03-19T21:20:34.628 INFO:teuthology.orchestra.run.smithi170.stderr: 2020-03-19T21:20:34.628 INFO:teuthology.orchestra.run.smithi170.stderr: Problem: conflicting requests 2020-03-19T21:20:34.628 INFO:teuthology.orchestra.run.smithi170.stderr: - nothing provides module(perl:5.26) needed by module perl-YAML:1.24:8010020190322131811:a5949e2e-0.x86_64 2020-03-19T21:20:34.660 INFO:teuthology.orchestra.run.smithi170.stdout:Dependencies resolved. 2020-03-19T21:20:34.662 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2020-03-19T21:20:34.662 INFO:teuthology.orchestra.run.smithi170.stdout: Package ArchVersion Repository Size 2020-03-19T21:20:34.662 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2020-03-19T21:20:34.662 INFO:teuthology.orchestra.run.smithi170.stdout:Removing: 2020-03-19T21:20:34.662 INFO:teuthology.orchestra.run.smithi170.stdout: librados2 x86_642:15.1.1-234.g787db24.el8 @ceph 16 M 2020-03-19T21:20:34.662 INFO:teuthology.orchestra.run.smithi170.stdout:Removing dependent packages: 2020-03-19T21:20:34.662 INFO:teuthology.orchestra.run.smithi170.stdout: qemu-kvm-block-rbd x86_6415:2.12.0-88.module+el8.1.0+5149+3ff2765e.2 @rhel-8-for-x86_64-appstream-rpms 34 k 2020-03-19T21:20:34.663 INFO:teuthology.orchestra.run.smithi170.stdout: rbd-fuse x86_642:15.1.1-234.g787db24.el8 @ceph 230 k 2020-03-19T21:20:34.663 INFO:teuthology.orchestra.run.smithi170.stdout: rbd-nbd x86_642:15.1.1-234.g787db24.el8 @ceph 526 k 2020-03-19T21:20:34.663 INFO:teuthology.orchestra.run.smithi170.stdout:Removing unused dependencies: 2020-03-19T21:20:34.663 INFO:teuthology.orchestra.run.smithi170.stdout: librbd1 x86_642:15.1.1-234.g787db24.el8 @ceph 8.6 M 2020-03-19T21:20:34.663 INFO:teuthology.orchestra.run.smithi170.stdout: lttng-ust x86_642.8.1-11.el8 @lab-extras 1.0 M 2020-03-19T21:20:34.663 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:20:34.663 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction Summary 2020-03-19T21:20:34.664 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2020-03-19T21:20:34.664 INFO:teuthology.orchestra.run.smithi170.stdout:Remove 6 Packages 2020-03-19T21:20:34.664 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:20:34.664 INFO:teuthology.orchestra.run.smithi170.stdout:Freed space: 26 M 2020-03-19T21:20:34.664 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction check 2020-03-19T21:20:34.666 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction check succeeded. 2020-03-19T21:20:34.667 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction test 2020-03-19T21:20:34.700 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction test succeeded. 2020-03-19T21:20:34.700 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction 2020-03-19T21:20:34.816 INFO:teuthology.orchestra.run.smithi170.stdout: Preparing : 1/1 2020-03-19T21:20:34.972 INFO:teuthology.orchestra.run.smithi170.stdout: Erasing : rbd-nbd-2:15.1.1-234.g787db24.el8.x86_64 1/6 2020-03-19T21:20:35.089 INFO:teuthology.orchestra.run.smithi170.stdout: Erasing : rbd-fuse-2:15.1.1-234.g787db24.el8.x86_64 2/6 2020-03-19T21:20:35.240 INFO:teuthology.orchestra.run.smithi170.stdout: Erasing : qemu-kvm-block-rbd-15:2.12.0-88.module+el8.1.0+5149+ 3/6 2020-03-19T21:20:35.240 INFO:teuthology.orchestra.run.smithi170.stdout: Erasing : librbd1-2:15.1.1-234.g787db24.el8.x86_64 4/6 2020-03-19T21:20:35.465 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: librbd1-2:15.1.1-234.g787db24.el8.x86_64 4/6 2020-03-19T21:20:35.465 INFO:teuthology.orchestra.run.smithi170.stdout: Erasing : librados2-2:15.1.1-234.g787db24.el8.x86_64 5/6 2020-03-19T21:20:35.465 INFO:teuthology.orchestra.run.smithi170.stdout:warning: file /etc/ceph: remove failed: No such file or directory 2020-03-19T21:20:35.466 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:20:35.658 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: librados2-2:15.1.1-234.g787db24.el8.x86_64 5/6 2020-03-19T21:20:35.658 INFO:teuthology.orchestra.run.smithi170.stdout: Erasing : lttng-ust-2.8.1-11.el8.x86_64 6/6 2020-03-19T21:20:36.084 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 6/6 2020-03-19T21:20:36.084 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librados2-2:15.1.1-234.g787db24.el8.x86_64 1/6 2020-03-19T21:20:36.084 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librbd1-2:15.1.1-234.g787db24.el8.x86_64 2/6 2020-03-19T21:20:36.084 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 3/6 2020-03-19T21:20:36.084 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : qemu-kvm-block-rbd-15:2.12.0-88.module+el8.1.0+5149+ 4/6 2020-03-19T21:20:36.085 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : rbd-fuse-2:15.1.1-234.g787db24.el8.x86_64 5/6 2020-03-19T21:20:38.147 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : rbd-nbd-2:15.1.1-234.g787db24.el8.x86_64 6/6 2020-03-19T21:20:38.147 INFO:teuthology.orchestra.run.smithi170.stdout:Installed products updated. 2020-03-19T21:20:38.329 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:20:38.329 INFO:teuthology.orchestra.run.smithi170.stdout:Removed: 2020-03-19T21:20:38.330 INFO:teuthology.orchestra.run.smithi170.stdout: librados2-2:15.1.1-234.g787db24.el8.x86_64 2020-03-19T21:20:38.330 INFO:teuthology.orchestra.run.smithi170.stdout: qemu-kvm-block-rbd-15:2.12.0-88.module+el8.1.0+5149+3ff2765e.2.x86_64 2020-03-19T21:20:38.330 INFO:teuthology.orchestra.run.smithi170.stdout: rbd-fuse-2:15.1.1-234.g787db24.el8.x86_64 2020-03-19T21:20:38.330 INFO:teuthology.orchestra.run.smithi170.stdout: rbd-nbd-2:15.1.1-234.g787db24.el8.x86_64 2020-03-19T21:20:38.330 INFO:teuthology.orchestra.run.smithi170.stdout: librbd1-2:15.1.1-234.g787db24.el8.x86_64 2020-03-19T21:20:38.331 INFO:teuthology.orchestra.run.smithi170.stdout: lttng-ust-2.8.1-11.el8.x86_64 2020-03-19T21:20:38.331 INFO:teuthology.orchestra.run.smithi170.stdout: 2020-03-19T21:20:38.331 INFO:teuthology.orchestra.run.smithi170.stdout:Complete! 2020-03-19T21:20:38.814 INFO:teuthology.orchestra.run.smithi170.stdout:Updating Subscription Management repositories. 2020-03-19T21:20:39.345 INFO:teuthology.orchestra.run.smithi170.stderr:Modular dependency problem: 2020-03-19T21:20:39.346 INFO:teuthology.orchestra.run.smithi170.stderr: 2020-03-19T21:20:39.346 INFO:teuthology.orchestra.run.smithi170.stderr: Problem: conflicting requests 2020-03-19T21:20:39.346 INFO:teuthology.orchestra.run.smithi170.stderr: - nothing provides module(perl:5.26) needed by module perl-YAML:1.24:8010020190322131811:a5949e2e-0.x86_64 2020-03-19T21:20:39.346 INFO:teuthology.orchestra.run.smithi170.stdout:No match for argument: librbd1 2020-03-19T21:20:39.347 INFO:teuthology.orchestra.run.smithi170.stderr:No packages marked for removal. 2020-03-19T21:20:39.351 INFO:teuthology.orchestra.run.smithi170.stdout:Dependencies resolved. 2020-03-19T21:20:39.352 INFO:teuthology.orchestra.run.smithi170.stdout:Nothing to do. 2020-03-19T21:20:39.352 INFO:teuthology.orchestra.run.smithi170.stdout:Complete! 2020-03-19T21:20:39.823 INFO:teuthology.orchestra.run.smithi170.stdout:Updating Subscription Management repositories. 2020-03-19T21:20:40.353 INFO:teuthology.orchestra.run.smithi170.stderr:Modular dependency problem: 2020-03-19T21:20:40.353 INFO:teuthology.orchestra.run.smithi170.stderr: 2020-03-19T21:20:40.353 INFO:teuthology.orchestra.run.smithi170.stderr: Problem: conflicting requests 2020-03-19T21:20:40.353 INFO:teuthology.orchestra.run.smithi170.stderr: - nothing provides module(perl:5.26) needed by module perl-YAML:1.24:8010020190322131811:a5949e2e-0.x86_64 2020-03-19T21:20:40.354 INFO:teuthology.orchestra.run.smithi170.stdout:No match for argument: python3-rados 2020-03-19T21:20:40.354 INFO:teuthology.orchestra.run.smithi170.stderr:No packages marked for removal. 2020-03-19T21:20:40.358 INFO:teuthology.orchestra.run.smithi170.stdout:Dependencies resolved. 2020-03-19T21:20:40.359 INFO:teuthology.orchestra.run.smithi170.stdout:Nothing to do. 2020-03-19T21:20:40.360 INFO:teuthology.orchestra.run.smithi170.stdout:Complete! 2020-03-19T21:20:40.804 INFO:teuthology.orchestra.run.smithi170.stdout:Updating Subscription Management repositories. 2020-03-19T21:20:41.325 INFO:teuthology.orchestra.run.smithi170.stderr:Modular dependency problem: 2020-03-19T21:20:41.326 INFO:teuthology.orchestra.run.smithi170.stderr: 2020-03-19T21:20:41.326 INFO:teuthology.orchestra.run.smithi170.stderr: Problem: conflicting requests 2020-03-19T21:20:41.326 INFO:teuthology.orchestra.run.smithi170.stderr: - nothing provides module(perl:5.26) needed by module perl-YAML:1.24:8010020190322131811:a5949e2e-0.x86_64 2020-03-19T21:20:41.326 INFO:teuthology.orchestra.run.smithi170.stdout:No match for argument: python3-rgw 2020-03-19T21:20:41.327 INFO:teuthology.orchestra.run.smithi170.stderr:No packages marked for removal. 2020-03-19T21:20:41.331 INFO:teuthology.orchestra.run.smithi170.stdout:Dependencies resolved. 2020-03-19T21:20:41.332 INFO:teuthology.orchestra.run.smithi170.stdout:Nothing to do. 2020-03-19T21:20:41.332 INFO:teuthology.orchestra.run.smithi170.stdout:Complete! 2020-03-19T21:20:41.804 INFO:teuthology.orchestra.run.smithi170.stdout:Updating Subscription Management repositories. 2020-03-19T21:20:42.290 INFO:teuthology.orchestra.run.smithi170.stderr:Modular dependency problem: 2020-03-19T21:20:42.291 INFO:teuthology.orchestra.run.smithi170.stderr: 2020-03-19T21:20:42.291 INFO:teuthology.orchestra.run.smithi170.stderr: Problem: conflicting requests 2020-03-19T21:20:42.291 INFO:teuthology.orchestra.run.smithi170.stderr: - nothing provides module(perl:5.26) needed by module perl-YAML:1.24:8010020190322131811:a5949e2e-0.x86_64 2020-03-19T21:20:42.292 INFO:teuthology.orchestra.run.smithi170.stdout:No match for argument: python3-cephfs 2020-03-19T21:20:42.292 INFO:teuthology.orchestra.run.smithi170.stderr:No packages marked for removal. 2020-03-19T21:20:42.296 INFO:teuthology.orchestra.run.smithi170.stdout:Dependencies resolved. 2020-03-19T21:20:42.297 INFO:teuthology.orchestra.run.smithi170.stdout:Nothing to do. 2020-03-19T21:20:42.297 INFO:teuthology.orchestra.run.smithi170.stdout:Complete! 2020-03-19T21:20:42.752 INFO:teuthology.orchestra.run.smithi170.stdout:Updating Subscription Management repositories. 2020-03-19T21:20:43.255 INFO:teuthology.orchestra.run.smithi170.stderr:Modular dependency problem: 2020-03-19T21:20:43.255 INFO:teuthology.orchestra.run.smithi170.stderr: 2020-03-19T21:20:43.255 INFO:teuthology.orchestra.run.smithi170.stderr: Problem: conflicting requests 2020-03-19T21:20:43.255 INFO:teuthology.orchestra.run.smithi170.stderr: - nothing provides module(perl:5.26) needed by module perl-YAML:1.24:8010020190322131811:a5949e2e-0.x86_64 2020-03-19T21:20:43.256 INFO:teuthology.orchestra.run.smithi170.stdout:No match for argument: python3-rbd 2020-03-19T21:20:43.256 INFO:teuthology.orchestra.run.smithi170.stderr:No packages marked for removal. 2020-03-19T21:20:43.260 INFO:teuthology.orchestra.run.smithi170.stdout:Dependencies resolved. 2020-03-19T21:20:43.261 INFO:teuthology.orchestra.run.smithi170.stdout:Nothing to do. 2020-03-19T21:20:43.262 INFO:teuthology.orchestra.run.smithi170.stdout:Complete! 2020-03-19T21:20:43.735 INFO:teuthology.orchestra.run.smithi170.stdout:Updating Subscription Management repositories. 2020-03-19T21:20:44.228 INFO:teuthology.orchestra.run.smithi170.stderr:Modular dependency problem: 2020-03-19T21:20:44.228 INFO:teuthology.orchestra.run.smithi170.stderr: 2020-03-19T21:20:44.228 INFO:teuthology.orchestra.run.smithi170.stderr: Problem: conflicting requests 2020-03-19T21:20:44.228 INFO:teuthology.orchestra.run.smithi170.stderr: - nothing provides module(perl:5.26) needed by module perl-YAML:1.24:8010020190322131811:a5949e2e-0.x86_64 2020-03-19T21:20:44.229 INFO:teuthology.orchestra.run.smithi170.stdout:No match for argument: rbd-fuse 2020-03-19T21:20:44.229 INFO:teuthology.orchestra.run.smithi170.stderr:No packages marked for removal. 2020-03-19T21:20:44.233 INFO:teuthology.orchestra.run.smithi170.stdout:Dependencies resolved. 2020-03-19T21:20:44.234 INFO:teuthology.orchestra.run.smithi170.stdout:Nothing to do. 2020-03-19T21:20:44.235 INFO:teuthology.orchestra.run.smithi170.stdout:Complete! 2020-03-19T21:20:44.676 INFO:teuthology.orchestra.run.smithi170.stdout:Updating Subscription Management repositories. 2020-03-19T21:20:45.235 INFO:teuthology.orchestra.run.smithi170.stderr:Modular dependency problem: 2020-03-19T21:20:45.235 INFO:teuthology.orchestra.run.smithi170.stderr: 2020-03-19T21:20:45.235 INFO:teuthology.orchestra.run.smithi170.stderr: Problem: conflicting requests 2020-03-19T21:20:45.235 INFO:teuthology.orchestra.run.smithi170.stderr: - nothing provides module(perl:5.26) needed by module perl-YAML:1.24:8010020190322131811:a5949e2e-0.x86_64 2020-03-19T21:20:45.236 INFO:teuthology.orchestra.run.smithi170.stdout:No match for argument: rbd-mirror 2020-03-19T21:20:45.236 INFO:teuthology.orchestra.run.smithi170.stderr:No packages marked for removal. 2020-03-19T21:20:45.241 INFO:teuthology.orchestra.run.smithi170.stdout:Dependencies resolved. 2020-03-19T21:20:45.242 INFO:teuthology.orchestra.run.smithi170.stdout:Nothing to do. 2020-03-19T21:20:45.242 INFO:teuthology.orchestra.run.smithi170.stdout:Complete! 2020-03-19T21:20:45.716 INFO:teuthology.orchestra.run.smithi170.stdout:Updating Subscription Management repositories. 2020-03-19T21:20:46.253 INFO:teuthology.orchestra.run.smithi170.stderr:Modular dependency problem: 2020-03-19T21:20:46.253 INFO:teuthology.orchestra.run.smithi170.stderr: 2020-03-19T21:20:46.253 INFO:teuthology.orchestra.run.smithi170.stderr: Problem: conflicting requests 2020-03-19T21:20:46.253 INFO:teuthology.orchestra.run.smithi170.stderr: - nothing provides module(perl:5.26) needed by module perl-YAML:1.24:8010020190322131811:a5949e2e-0.x86_64 2020-03-19T21:20:46.254 INFO:teuthology.orchestra.run.smithi170.stdout:No match for argument: rbd-nbd 2020-03-19T21:20:46.254 INFO:teuthology.orchestra.run.smithi170.stderr:No packages marked for removal. 2020-03-19T21:20:46.258 INFO:teuthology.orchestra.run.smithi170.stdout:Dependencies resolved. 2020-03-19T21:20:46.259 INFO:teuthology.orchestra.run.smithi170.stdout:Nothing to do. 2020-03-19T21:20:46.260 INFO:teuthology.orchestra.run.smithi170.stdout:Complete! 2020-03-19T21:20:46.434 INFO:teuthology.orchestra.run.smithi170:> sudo yum clean all 2020-03-19T21:20:46.760 INFO:teuthology.orchestra.run.smithi170.stdout:Updating Subscription Management repositories. 2020-03-19T21:20:47.148 INFO:teuthology.orchestra.run.smithi170.stdout:52 files removed 2020-03-19T21:20:47.191 INFO:teuthology.orchestra.run.smithi170:> sudo lsb_release -is 2020-03-19T21:20:47.226 INFO:teuthology.orchestra.run.smithi170.stdout:RedHatEnterprise 2020-03-19T21:20:47.227 DEBUG:teuthology.misc:System to be installed: RedHatEnterprise 2020-03-19T21:20:47.228 INFO:teuthology.orchestra.run.smithi170:> sudo rm -f /etc/yum.repos.d/ceph.repo 2020-03-19T21:20:47.295 INFO:teuthology.orchestra.run.smithi170:> sudo yum clean expire-cache 2020-03-19T21:20:47.660 INFO:teuthology.orchestra.run.smithi170.stdout:Updating Subscription Management repositories. 2020-03-19T21:20:48.124 INFO:teuthology.orchestra.run.smithi170.stdout:Cache was expired 2020-03-19T21:20:48.124 INFO:teuthology.orchestra.run.smithi170.stdout:0 files removed 2020-03-19T21:20:48.168 DEBUG:teuthology.parallel:result is None 2020-03-19T21:20:48.168 INFO:teuthology.task.install:Removing ceph sources lists 2020-03-19T21:20:48.168 INFO:teuthology.orchestra.run.smithi170:> sudo lsb_release -is 2020-03-19T21:20:48.203 INFO:teuthology.orchestra.run.smithi170.stdout:RedHatEnterprise 2020-03-19T21:20:48.205 DEBUG:teuthology.misc:System to be installed: RedHatEnterprise 2020-03-19T21:20:48.205 INFO:teuthology.orchestra.run.smithi170:> sudo rm -f /etc/yum.repos.d/ceph.repo 2020-03-19T21:20:48.272 INFO:teuthology.orchestra.run.smithi170:> sudo mv -f /etc/yum/pluginconf.d/priorities.conf.orig /etc/yum/pluginconf.d/priorities.conf 2020-03-19T21:20:48.339 DEBUG:teuthology.parallel:result is None 2020-03-19T21:20:48.339 DEBUG:teuthology.run_tasks:Unwinding manager clock 2020-03-19T21:20:48.353 INFO:teuthology.task.clock:Checking final clock skew... 2020-03-19T21:20:48.353 INFO:teuthology.orchestra.run.smithi170:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2020-03-19T21:20:48.391 INFO:teuthology.orchestra.run.smithi170.stderr:bash: ntpq: command not found 2020-03-19T21:20:48.394 INFO:teuthology.orchestra.run.smithi170.stdout:210 Number of sources = 4 2020-03-19T21:20:48.395 INFO:teuthology.orchestra.run.smithi170.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2020-03-19T21:20:48.395 INFO:teuthology.orchestra.run.smithi170.stdout:=============================================================================== 2020-03-19T21:20:48.395 INFO:teuthology.orchestra.run.smithi170.stdout:^? hv01.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2020-03-19T21:20:48.395 INFO:teuthology.orchestra.run.smithi170.stdout:^? hv02.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2020-03-19T21:20:48.395 INFO:teuthology.orchestra.run.smithi170.stdout:^? hv03.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2020-03-19T21:20:48.395 INFO:teuthology.orchestra.run.smithi170.stdout:^* hv04.front.sepia.ceph.com 3 6 377 45 -52us[ -249us] +/- 36ms 2020-03-19T21:20:48.396 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2020-03-19T21:20:48.409 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2020-03-19T21:20:48.410 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2020-03-19T21:20:48.425 INFO:teuthology.orchestra.run.smithi170:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2020-03-19T21:20:48.494 INFO:teuthology.orchestra.run.smithi170:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av '\(comm="dmidecode"\|chronyd.service\|name="cephtest"\|scontext=system_u:system_r:nrpe_t:s0\|scontext=system_u:system_r:pcp_pmlogger_t\|scontext=system_u:system_r:pcp_pmcd_t:s0\|comm="rhsmd"\|scontext=system_u:system_r:syslogd_t:s0\|tcontext=system_u:system_r:nrpe_t:s0\|comm="updatedb"\|comm="smartd"\|comm="rhsmcertd-worke"\|comm="setroubleshootd"\|comm="rpm"\|tcontext=system_u:object_r:container_runtime_exec_t:s0\|scontext=system_u:system_r:logrotate_t:s0\)' 2020-03-19T21:20:48.523 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-03-19T21:20:48.523 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2020-03-19T21:20:48.536 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2020-03-19T21:20:48.562 INFO:teuthology.task.internal:Duration was 583.019516 seconds 2020-03-19T21:20:48.562 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2020-03-19T21:20:48.578 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2020-03-19T21:20:48.578 INFO:teuthology.orchestra.run.smithi170:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2020-03-19T21:20:48.628 INFO:teuthology.orchestra.run.smithi170.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2020-03-19T21:20:48.780 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2020-03-19T21:20:48.781 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi170.front.sepia.ceph.com 2020-03-19T21:20:48.781 INFO:teuthology.orchestra.run.smithi170:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/*.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2020-03-19T21:20:48.799 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2020-03-19T21:20:48.799 INFO:teuthology.orchestra.run.smithi170:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2020-03-19T21:20:48.873 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2020-03-19T21:20:48.887 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2020-03-19T21:20:48.887 INFO:teuthology.orchestra.run.smithi170:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2020-03-19T21:20:48.941 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2020-03-19T21:20:48.955 INFO:teuthology.orchestra.run.smithi170:> sudo sysctl -w kernel.core_pattern=core && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2020-03-19T21:20:49.007 INFO:teuthology.orchestra.run.smithi170.stdout:kernel.core_pattern = core 2020-03-19T21:20:49.009 INFO:teuthology.orchestra.run.smithi170:> test -e /home/ubuntu/cephtest/archive/coredump 2020-03-19T21:20:49.064 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-03-19T21:20:49.064 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2020-03-19T21:20:49.084 INFO:teuthology.task.internal:Transferring archived files... 2020-03-19T21:20:49.085 DEBUG:teuthology.misc:Transferring archived files from smithi170:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/sage-2020-03-19_17:25:48-rados-wip-sage-testing-2020-03-19-0855-distro-basic-smithi/4868524/remote/smithi170 2020-03-19T21:20:49.085 INFO:teuthology.orchestra.run.smithi170:> sudo tar cz -f - -C /home/ubuntu/cephtest/archive -- . 2020-03-19T21:20:49.142 INFO:teuthology.task.internal:Removing archive directory... 2020-03-19T21:20:49.142 INFO:teuthology.orchestra.run.smithi170:> rm -rf -- /home/ubuntu/cephtest/archive 2020-03-19T21:20:49.191 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2020-03-19T21:20:49.205 INFO:teuthology.task.internal:Not uploading archives. 2020-03-19T21:20:49.205 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2020-03-19T21:20:49.219 INFO:teuthology.task.internal:Tidying up after the test... 2020-03-19T21:20:49.220 INFO:teuthology.orchestra.run.smithi170:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2020-03-20T08:58:41.409 DEBUG:teuthology.exit:Got signal 15; running 2 handlers... 2020-03-20T08:58:41.426 DEBUG:teuthology.task.console_log:Killing console logger for smithi170 2020-03-20T08:58:41.426 DEBUG:teuthology.task.console_log:Killing console logger for smithi170 2020-03-20T08:58:41.427 DEBUG:teuthology.exit:Finished running handlers